Re: [Qemu-devel] [RFC V2 0/4] vfio: Introduce Live migration capability to vfio_mdev device

2017-07-30 Thread Tian, Kevin
> From: Zhang, Yulei
> Sent: Tuesday, May 9, 2017 3:59 PM
> 
> Summary
> 
> This series RFC would like to introduce the live migration capability
> to vfio_mdev device.
> 
> As currently vfio_mdev device don't support migration, we introduce
> a new vfio subtype region
> VFIO_REGION_SUBTYPE_INTEL_IGD_DEVICE_STATE
> for Intel vGPU device, during the vfio device initialization, the mdev
> device will be set to migratable if the new region exist.

Looking at your series, there is really nothing specific to vGPU or
even Intel vGPU regarding to device state save/restore...

> 
> The intention to add the new region is using it for vfio_mdev device
> status save and restore during the migration. The access to this region
> will be trapped and forward to the vfio_mdev device driver. And we use
> the first byte in the new region to control the running state of mdev
> device.
> 
> Meanwhile we add one new ioctl VFIO_IOMMU_GET_DIRTY_BITMAP to help
> do
> the mdev device dirty page synchronization.
> 
> So the vfio_mdev device migration sequence would be
> Source VM side:
>   start migration
>   |
>   V
>get the cpu state change callback, write to the
>subregion's first byte to stop the mdev device
>   |
>   V
>quary the dirty page bitmap from iommu container
>and add into qemu dirty list for synchronization
>   |
>   V
>save the deivce status into Qemufile which is
>  read from the vfio device subregion
> 
> Target VM side:
>  restore the mdev device after get the
>saved status context from Qemufile
>   |
>   V
>get the cpu state change callback
>write to subregion's first byte to
>   start the mdev device to put it in
>   running status
>   |
>   V
>   finish migration
> 
> V1->V2:
> Per Alex's suggestion:
> 1. use device subtype region instead of VFIO PCI fixed region.
> 2. remove unnecessary ioctl, use the first byte of subregion to
>control the running state of mdev device.
> 3. for dirty page synchronization, implement the interface with
>VFIOContainer instead of vfio pci device.
> 
> Yulei Zhang (4):
>   vfio: introduce a new VFIO sub region for mdev device migration
> support
>   vfio: Add vm status change callback to stop/restart the mdev device
>   vfio: Add struct vfio_vmstate_info to introduce put/get callback
> funtion for vfio device status save/restore
>   vifo: introduce new VFIO ioctl VFIO_IOMMU_GET_DIRTY_BITMAP
> 
>  hw/vfio/common.c  |  32 +
>  hw/vfio/pci.c | 164
> +-
>  hw/vfio/pci.h |   1 +
>  include/hw/vfio/vfio-common.h |   1 +
>  linux-headers/linux/vfio.h|  26 ++-
>  5 files changed, 220 insertions(+), 4 deletions(-)
> 
> --
> 2.7.4




Re: [Qemu-devel] [RFC V2 0/4] vfio: Introduce Live migration capability to vfio_mdev device

2017-07-30 Thread no-reply
Hi,

This series seems to have some coding style problems. See output below for
more information:

Subject: [Qemu-devel] [RFC V2 0/4] vfio: Introduce Live migration capability to 
vfio_mdev device
Message-id: 1494316727-15518-1-git-send-email-yulei.zh...@intel.com
Type: series

=== TEST SCRIPT BEGIN ===
#!/bin/bash

BASE=base
n=1
total=$(git log --oneline $BASE.. | wc -l)
failed=0

git config --local diff.renamelimit 0
git config --local diff.renames True

commits="$(git log --format=%H --reverse $BASE..)"
for c in $commits; do
echo "Checking PATCH $n/$total: $(git log -n 1 --format=%s $c)..."
if ! git show $c --format=email | ./scripts/checkpatch.pl --mailback -; then
failed=1
echo
fi
n=$((n+1))
done

exit $failed
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
e72290dd77 vifo: introduce new VFIO ioctl VFIO_IOMMU_GET_DIRTY_BITMAP
e63ec9785b vfio: Add struct vfio_vmstate_info to introduce put/get callback 
funtion for vfio device status save/restore
196e200d3c vfio: Add vm status change callback to stop/restart the mdev device
bd9ff9bed2 vfio: introduce a new VFIO sub region for mdev device migration 
support

=== OUTPUT BEGIN ===
Checking PATCH 1/4: vfio: introduce a new VFIO sub region for mdev device 
migration support...
WARNING: line over 80 characters
#34: FILE: hw/vfio/pci.c:2822:
+   memcpy(&vdev->device_state, device_state, sizeof(struct 
vfio_region_info));

ERROR: code indent should never use tabs
#34: FILE: hw/vfio/pci.c:2822:
+^Imemcpy(&vdev->device_state, device_state, sizeof(struct vfio_region_info));$

ERROR: code indent should never use tabs
#35: FILE: hw/vfio/pci.c:2823:
+^Ig_free(device_state);$

ERROR: initializer for struct VMStateDescription should normally be const
#47: FILE: hw/vfio/pci.c:3008:
+static VMStateDescription vfio_pci_vmstate = {

total: 3 errors, 1 warnings, 51 lines checked

Your patch has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

Checking PATCH 2/4: vfio: Add vm status change callback to stop/restart the 
mdev device...
WARNING: line over 80 characters
#49: FILE: hw/vfio/pci.c:2975:
+if (pwrite(vdev->vbasedev.fd, &dev_state, sz, vdev->device_state.offset) 
!= sz) {

ERROR: Error messages should not contain newlines
#50: FILE: hw/vfio/pci.c:2976:
+error_report("vfio: Failed to %s device\n", running ? "start" : 
"stop");

total: 1 errors, 1 warnings, 53 lines checked

Your patch has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

Checking PATCH 3/4: vfio: Add struct vfio_vmstate_info to introduce put/get 
callback funtion for vfio device status save/restore...
WARNING: line over 80 characters
#33: FILE: hw/vfio/pci.c:2983:
+static int vfio_device_put(QEMUFile *f, void *pv, size_t size, VMStateField 
*field,

ERROR: spaces required around that '*' (ctx:VxV)
#44: FILE: hw/vfio/pci.c:2994:
+bar_cfg = pci_default_read_config(pdev, PCI_BASE_ADDRESS_0 + i*4, 4);
   ^

WARNING: line over 80 characters
#51: FILE: hw/vfio/pci.c:3001:
+msi_lo = pci_default_read_config(pdev, pdev->msi_cap + PCI_MSI_ADDRESS_LO, 
4);

WARNING: line over 80 characters
#55: FILE: hw/vfio/pci.c:3005:
+msi_hi = pci_default_read_config(pdev, pdev->msi_cap + 
PCI_MSI_ADDRESS_HI, 4);

WARNING: line over 80 characters
#60: FILE: hw/vfio/pci.c:3010:
+ pdev->msi_cap + (msi_64bit ? PCI_MSI_DATA_64 : 
PCI_MSI_DATA_32), 2);

ERROR: Error messages should not contain newlines
#65: FILE: hw/vfio/pci.c:3015:
+error_report("vfio: Failed to allocate memory for migrate\n");

ERROR: Error messages should not contain newlines
#71: FILE: hw/vfio/pci.c:3021:
+error_report("vfio: Failed to read Device State Region\n");

ERROR: braces {} are necessary for all arms of this statement
#78: FILE: hw/vfio/pci.c:3028:
+if (buf)
[...]

ERROR: g_free(NULL) is safe this check is probably not required
#79: FILE: hw/vfio/pci.c:3029:
+if (buf)
+g_free(buf);

WARNING: line over 80 characters
#84: FILE: hw/vfio/pci.c:3034:
+static int vfio_device_get(QEMUFile *f, void *pv, size_t size, VMStateField 
*field)

ERROR: spaces required around that '*' (ctx:VxV)
#99: FILE: hw/vfio/pci.c:3049:
+vfio_pci_write_config(pdev, PCI_BASE_ADDRESS_0 + i*4, bar_cfg, 4);
   ^

WARNING: line over 80 characters
#117: FILE: hw/vfio/pci.c:3067:
+vfio_pci_write_config(pdev, pdev->msi_cap + PCI_MSI_ADDRESS_HI, 
msi_hi, 4);

WARNING: line over 80 characters
#121: FILE: hw/vfio/pci.c:3071:
+  pdev->msi_cap + (msi_64bit ? PCI_MSI_DATA_64 : 
PCI_MSI_DATA_32),

ERROR: Error messages should not contain newlines
#129: FILE: hw/vfio/pci.c:3079:
+e

Re: [Qemu-devel] [for-2.11 PATCH 24/26] spapr: allow guest to update the XICS phandle

2017-07-30 Thread David Gibson
On Fri, Jul 28, 2017 at 08:20:57AM +0200, Thomas Huth wrote:
> On 28.07.2017 06:02, David Gibson wrote:
> > On Tue, Jul 25, 2017 at 08:03:06PM +0200, Greg Kurz wrote:
> >> The "phandle" property of the XICS node is referenced by the 
> >> "interrupt-map"
> >> property of each PHB node. This is used by the guest OS to setup IRQs for
> >> all PCI devices.
> >>
> >> QEMU uses an arbitrary value (0x) for this phandle, but SLOF converts
> >> this value to a SLOF specific one, which is then presented to the guest OS.
> >>
> >> This patches introduces the new KVMPPC_H_UPDATE_PHANDLE hcall, which is 
> >> used
> >> by SLOF to communicate the patched phandle value back to QEMU. This value
> >> is then cached and preserved accross migration until machine reset.
> >>
> >> This is required to be able to support PHB hotplug.
> >>
> >> Note, that SLOF already has some code to call KVMPPC_H_RTAS_UPDATE, so we
> >> have to introduce its number even if QEMU currently doesn't implement it.
> >>
> >> Suggested-by: Thomas Huth 
> >> Signed-off-by: Greg Kurz 
> > 
> > Ugh.  I really, really hope we can avoid this, though I don't
> > immediately see how.  Having to have two way communication between
> > qemu and SLOF about the device tree contents just seems like opening
> > the door to endless complexities.
> > 
> > This is basically a consequence of the fact that both qemu and partly
> > responsible for constructing the device tree for the guest, and that's
> > not easy to avoid.
> > 
> > Hrm.. Thomas, I know it's not really the OF way, but would it be
> > feasible to change SLOF to use the phandles as supplied by qemu rather
> > than creating its own?
> 
> I don't see a way to do this in an easy, clean, reasonable way. SLOF
> uses pointers to internal structures as phandles all over the place. You
> likely can't replace that so easily without rewriting half of the whole
> device tree related code in SLOF, I guess...

Dang, that's what I suspected.

Just to be clear the phandles are used directly as raw pointers?
There's not even some lookup macro we could change?

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature


[Qemu-devel] [RFC V2 4/4] vifo: introduce new VFIO ioctl VFIO_IOMMU_GET_DIRTY_BITMAP

2017-07-30 Thread Yulei Zhang
New VFIO ioctl VFIO_IOMMU_GET_DIRTY_BITMAP is used to fetch the
bitmap of pinned memory in iommu container, we need copy those
memory to the target during the migration as they are dirtied by
mdev devices.

Signed-off-by: Yulei Zhang 
---
 hw/vfio/common.c   | 32 
 linux-headers/linux/vfio.h | 14 ++
 2 files changed, 46 insertions(+)

diff --git a/hw/vfio/common.c b/hw/vfio/common.c
index f3ba9b9..54d43d5 100644
--- a/hw/vfio/common.c
+++ b/hw/vfio/common.c
@@ -35,6 +35,7 @@
 #include "sysemu/kvm.h"
 #include "trace.h"
 #include "qapi/error.h"
+#include "exec/ram_addr.h"
 
 struct vfio_group_head vfio_group_list =
 QLIST_HEAD_INITIALIZER(vfio_group_list);
@@ -603,9 +604,40 @@ static void vfio_listener_region_del(MemoryListener 
*listener,
 }
 }
 
+static void vfio_log_sync(MemoryListener *listener,
+  MemoryRegionSection *section)
+{
+VFIOContainer *container = container_of(listener, VFIOContainer, listener);
+VFIOGroup *group = QLIST_FIRST(&container->group_list);
+VFIODevice *vbasedev;
+QLIST_FOREACH(vbasedev, &group->device_list, next) {
+   if (vbasedev->device_state == VFIO_DEVICE_START)
+   return;
+}
+
+struct vfio_iommu_get_dirty_bitmap *d;
+ram_addr_t size = int128_get64(section->size);
+unsigned long page_nr = size >> TARGET_PAGE_BITS;
+unsigned long bitmap_size = (BITS_TO_LONGS(page_nr) + 1) * sizeof(unsigned 
long);
+d = g_malloc0(sizeof(*d) + bitmap_size);
+d->start_addr = section->offset_within_address_space;
+d->page_nr = page_nr;
+
+if (ioctl(container->fd, VFIO_IOMMU_GET_DIRTY_BITMAP, d)) {
+error_report("vfio: Failed to fetch dirty pages for migration\n");
+goto exit;
+}
+
+cpu_physical_memory_set_dirty_lebitmap((unsigned long*)&d->dirty_bitmap, 
d->start_addr, d->page_nr);
+
+exit:
+g_free(d);
+}
+
 static const MemoryListener vfio_memory_listener = {
 .region_add = vfio_listener_region_add,
 .region_del = vfio_listener_region_del,
+.log_sync = vfio_log_sync,
 };
 
 static void vfio_listener_release(VFIOContainer *container)
diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h
index dbbe7e1..cf3d163 100644
--- a/linux-headers/linux/vfio.h
+++ b/linux-headers/linux/vfio.h
@@ -553,6 +553,20 @@ struct vfio_iommu_type1_dma_unmap {
 #define VFIO_IOMMU_ENABLE  _IO(VFIO_TYPE, VFIO_BASE + 15)
 #define VFIO_IOMMU_DISABLE _IO(VFIO_TYPE, VFIO_BASE + 16)
 
+/**
+ * VFIO_IOMMU_GET_DIRTY_BITMAP - _IOW(VFIO_TYPE, VFIO_BASE + 17,
+ * struct vfio_iommu_get_dirty_bitmap)
+ *
+ * Return: 0 on success, -errno on failure.
+ */
+struct vfio_iommu_get_dirty_bitmap{
+   __u64  start_addr;
+   __u64  page_nr;
+   __u8   dirty_bitmap[];
+};
+
+#define VFIO_IOMMU_GET_DIRTY_BITMAP _IO(VFIO_TYPE, VFIO_BASE + 17)
+
 /*  Additional API for SPAPR TCE (Server POWERPC) IOMMU  */
 
 /*
-- 
2.7.4




[Qemu-devel] [RFC V2 2/4] vfio: Add vm status change callback to stop/restart the mdev device

2017-07-30 Thread Yulei Zhang
VM status change handler is added to change the vfio pci device
status during the migration, write the demanded device status
to the DEVICE STATUS subregion to stop the device on the source side
before fetch its status and start the deivce on the target side
after restore its status.

Signed-off-by: Yulei Zhang 
---
 hw/vfio/pci.c | 19 +++
 include/hw/vfio/vfio-common.h |  1 +
 linux-headers/linux/vfio.h|  3 +++
 3 files changed, 23 insertions(+)

diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
index 21a5cef..753da80 100644
--- a/hw/vfio/pci.c
+++ b/hw/vfio/pci.c
@@ -38,6 +38,7 @@
 static void vfio_disable_interrupts(VFIOPCIDevice *vdev);
 static void vfio_mmap_set_enabled(VFIOPCIDevice *vdev, bool enabled);
 static VMStateDescription vfio_pci_vmstate;
+static void vfio_vm_change_state_handler(void *pv, int running, RunState 
state);
 
 /*
  * Disabling BAR mmaping can be slow, but toggling it around INTx can
@@ -2858,6 +2859,7 @@ static void vfio_realize(PCIDevice *pdev, Error **errp)
 vfio_register_err_notifier(vdev);
 vfio_register_req_notifier(vdev);
 vfio_setup_resetfn_quirk(vdev);
+qemu_add_vm_change_state_handler(vfio_vm_change_state_handler, vdev);
 
 return;
 
@@ -2940,6 +2942,23 @@ post_reset:
 vfio_pci_post_reset(vdev);
 }
 
+static void vfio_vm_change_state_handler(void *pv, int running, RunState state)
+{
+VFIOPCIDevice *vdev = pv;
+VFIODevice *vbasedev = &vdev->vbasedev;
+uint8_t dev_state;
+uint8_t sz = 1;
+
+dev_state = running ? VFIO_DEVICE_START : VFIO_DEVICE_STOP;
+
+if (pwrite(vdev->vbasedev.fd, &dev_state, sz, vdev->device_state.offset) 
!= sz) {
+error_report("vfio: Failed to %s device\n", running ? "start" : 
"stop");
+return;
+}
+
+vbasedev->device_state = dev_state;
+}
+
 static void vfio_instance_init(Object *obj)
 {
 PCIDevice *pci_dev = PCI_DEVICE(obj);
diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h
index c582de1..c4bab97 100644
--- a/include/hw/vfio/vfio-common.h
+++ b/include/hw/vfio/vfio-common.h
@@ -123,6 +123,7 @@ typedef struct VFIODevice {
 unsigned int num_irqs;
 unsigned int num_regions;
 unsigned int flags;
+bool device_state;
 } VFIODevice;
 
 struct VFIODeviceOps {
diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h
index e2c53bf..ae1b953 100644
--- a/linux-headers/linux/vfio.h
+++ b/linux-headers/linux/vfio.h
@@ -299,6 +299,9 @@ struct vfio_region_info_cap_type {
 #define VFIO_REGION_SUBTYPE_INTEL_IGD_LPC_CFG  (3)
 #define VFIO_REGION_SUBTYPE_INTEL_IGD_DEVICE_STATE (4)
 
+#define VFIO_DEVICE_START  0
+#define VFIO_DEVICE_STOP   1
+
 /**
  * VFIO_DEVICE_GET_IRQ_INFO - _IOWR(VFIO_TYPE, VFIO_BASE + 9,
  * struct vfio_irq_info)
-- 
2.7.4




[Qemu-devel] [RFC V2 3/4] vfio: Add struct vfio_vmstate_info to introduce put/get callback funtion for vfio device status save/restore

2017-07-30 Thread Yulei Zhang
Introduce vfio_device_put/vfio_device_get funtion for vfio device state
save/restore usage.

For VFIO pci device status migrate, on the source side with
funtion vfio_device_put to save the following states
1. pci configuration space addr0~addr5
2. pci configuration space msi_addr msi_data
3. pci device status fetch from device driver

And on the target side with funtion vfio_device_get to restore
the same states
1. re-setup the pci bar configuration
2. re-setup the pci device msi configuration
3. restore the pci device status

Signed-off-by: Yulei Zhang 
---
 hw/vfio/pci.c  | 132 +
 linux-headers/linux/vfio.h |   2 +
 2 files changed, 134 insertions(+)

diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
index 753da80..c0fc1d2 100644
--- a/hw/vfio/pci.c
+++ b/hw/vfio/pci.c
@@ -2959,6 +2959,118 @@ static void vfio_vm_change_state_handler(void *pv, int 
running, RunState state)
 vbasedev->device_state = dev_state;
 }
 
+static int vfio_device_put(QEMUFile *f, void *pv, size_t size, VMStateField 
*field,
+QJSON *vmdesc)
+{
+VFIOPCIDevice *vdev = pv;
+PCIDevice *pdev = &vdev->pdev;
+int sz = vdev->device_state.size - VFIO_DEVICE_STATE_OFFSET;
+uint8_t *buf = NULL;
+uint32_t msi_cfg, msi_lo, msi_hi, msi_data, bar_cfg, i;
+bool msi_64bit;
+
+for (i = 0; i < PCI_ROM_SLOT; i++) {
+bar_cfg = pci_default_read_config(pdev, PCI_BASE_ADDRESS_0 + i*4, 4);
+qemu_put_be32(f, bar_cfg);
+}
+
+msi_cfg = pci_default_read_config(pdev, pdev->msi_cap + PCI_MSI_FLAGS, 2);
+msi_64bit = !!(msi_cfg & PCI_MSI_FLAGS_64BIT);
+
+msi_lo = pci_default_read_config(pdev, pdev->msi_cap + PCI_MSI_ADDRESS_LO, 
4);
+qemu_put_be32(f, msi_lo);
+
+if (msi_64bit) {
+msi_hi = pci_default_read_config(pdev, pdev->msi_cap + 
PCI_MSI_ADDRESS_HI, 4);
+qemu_put_be32(f, msi_hi);
+}
+
+msi_data = pci_default_read_config(pdev,
+ pdev->msi_cap + (msi_64bit ? PCI_MSI_DATA_64 : 
PCI_MSI_DATA_32), 2);
+qemu_put_be32(f, msi_data);
+
+buf = g_malloc(sz);
+if (buf == NULL) {
+error_report("vfio: Failed to allocate memory for migrate\n");
+goto exit;
+}
+
+if (pread(vdev->vbasedev.fd, buf, sz,
+  vdev->device_state.offset + VFIO_DEVICE_STATE_OFFSET) != sz) {
+error_report("vfio: Failed to read Device State Region\n");
+goto exit;
+}
+
+qemu_put_buffer(f, buf, sz);
+
+exit:
+if (buf)
+g_free(buf);
+
+return 0;
+}
+
+static int vfio_device_get(QEMUFile *f, void *pv, size_t size, VMStateField 
*field)
+{
+VFIOPCIDevice *vdev = pv;
+PCIDevice *pdev = &vdev->pdev;
+int sz = vdev->device_state.size - VFIO_DEVICE_STATE_OFFSET;
+uint8_t *buf = NULL;
+uint32_t ctl, msi_lo, msi_hi, msi_data, bar_cfg, i;
+bool msi_64bit;
+
+/* retore pci bar configuration */
+ctl = pci_default_read_config(pdev, PCI_COMMAND, 2);
+vfio_pci_write_config(pdev, PCI_COMMAND,
+  ctl & (!(PCI_COMMAND_IO | PCI_COMMAND_MEMORY)), 2);
+for (i = 0; i < PCI_ROM_SLOT; i++) {
+bar_cfg = qemu_get_be32(f);
+vfio_pci_write_config(pdev, PCI_BASE_ADDRESS_0 + i*4, bar_cfg, 4);
+}
+vfio_pci_write_config(pdev, PCI_COMMAND,
+  ctl | PCI_COMMAND_IO | PCI_COMMAND_MEMORY, 2);
+
+/* restore msi configuration */
+ctl = pci_default_read_config(pdev, pdev->msi_cap + PCI_MSI_FLAGS, 2);
+msi_64bit = !!(ctl & PCI_MSI_FLAGS_64BIT);
+
+vfio_pci_write_config(&vdev->pdev,
+  pdev->msi_cap + PCI_MSI_FLAGS,
+  ctl & (!PCI_MSI_FLAGS_ENABLE), 2);
+
+msi_lo = qemu_get_be32(f);
+vfio_pci_write_config(pdev, pdev->msi_cap + PCI_MSI_ADDRESS_LO, msi_lo, 4);
+
+if (msi_64bit) {
+msi_hi = qemu_get_be32(f);
+vfio_pci_write_config(pdev, pdev->msi_cap + PCI_MSI_ADDRESS_HI, 
msi_hi, 4);
+}
+msi_data = qemu_get_be32(f);
+vfio_pci_write_config(pdev,
+  pdev->msi_cap + (msi_64bit ? PCI_MSI_DATA_64 : 
PCI_MSI_DATA_32),
+  msi_data, 2);
+
+vfio_pci_write_config(&vdev->pdev, pdev->msi_cap + PCI_MSI_FLAGS,
+  ctl | PCI_MSI_FLAGS_ENABLE, 2);
+
+buf = g_malloc(sz);
+if (buf == NULL) {
+error_report("vfio: Failed to allocate memory for migrate\n");
+return -1;
+}
+
+qemu_get_buffer(f, buf, sz);
+if (pwrite(vdev->vbasedev.fd, buf, sz,
+   vdev->device_state.offset + VFIO_DEVICE_STATE_OFFSET) != sz) {
+error_report("vfio: Failed to write Device State Region\n");
+return -1;
+}
+
+if (buf)
+   g_free(buf);
+return 0;
+}
+
 static void vfio_instance_init(Object *obj)
 {
 PCIDevice *pci_dev = PCI_DEVICE(obj);
@@ -3003,9 +3115,29 @@ static Property vfio_pci_dev_properties[] = {
 DEFINE_PROP_END_OF_LIST(),
 };
 
+static V

[Qemu-devel] [RFC V2 1/4] vfio: introduce a new VFIO sub region for mdev device migration support

2017-07-30 Thread Yulei Zhang
New VFIO sub region VFIO_REGION_SUBTYPE_INTEL_IGD_DEVICE_STATE is added
to fetch and restore the status of mdev device vGPU during the live migration.

Signed-off-by: Yulei Zhang 
---
 hw/vfio/pci.c  | 13 -
 hw/vfio/pci.h  |  1 +
 linux-headers/linux/vfio.h |  7 ---
 3 files changed, 17 insertions(+), 4 deletions(-)

diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
index 03a3d01..21a5cef 100644
--- a/hw/vfio/pci.c
+++ b/hw/vfio/pci.c
@@ -37,6 +37,7 @@
 
 static void vfio_disable_interrupts(VFIOPCIDevice *vdev);
 static void vfio_mmap_set_enabled(VFIOPCIDevice *vdev, bool enabled);
+static VMStateDescription vfio_pci_vmstate;
 
 /*
  * Disabling BAR mmaping can be slow, but toggling it around INTx can
@@ -2792,6 +2793,16 @@ static void vfio_realize(PCIDevice *pdev, Error **errp)
 vfio_vga_quirk_setup(vdev);
 }
 
+struct vfio_region_info *device_state;
+/* device state region setup */
+if (!vfio_get_dev_region_info(&vdev->vbasedev,
+VFIO_REGION_TYPE_PCI_VENDOR_TYPE | PCI_VENDOR_ID_INTEL,
+VFIO_REGION_SUBTYPE_INTEL_IGD_DEVICE_STATE, &device_state)) {
+   memcpy(&vdev->device_state, device_state, sizeof(struct 
vfio_region_info));
+   g_free(device_state);
+vfio_pci_vmstate.unmigratable = 0;
+}
+
 for (i = 0; i < PCI_ROM_SLOT; i++) {
 vfio_bar_quirk_setup(vdev, i);
 }
@@ -2973,7 +2984,7 @@ static Property vfio_pci_dev_properties[] = {
 DEFINE_PROP_END_OF_LIST(),
 };
 
-static const VMStateDescription vfio_pci_vmstate = {
+static VMStateDescription vfio_pci_vmstate = {
 .name = "vfio-pci",
 .unmigratable = 1,
 };
diff --git a/hw/vfio/pci.h b/hw/vfio/pci.h
index a8366bb..6a1d26e 100644
--- a/hw/vfio/pci.h
+++ b/hw/vfio/pci.h
@@ -116,6 +116,7 @@ typedef struct VFIOPCIDevice {
 VFIOBAR bars[PCI_NUM_REGIONS - 1]; /* No ROM */
 VFIOVGA *vga; /* 0xa, 0x3b0, 0x3c0 */
 void *igd_opregion;
+struct vfio_region_info device_state;
 PCIHostDeviceAddress host;
 EventNotifier err_notifier;
 EventNotifier req_notifier;
diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h
index 531cb2e..e2c53bf 100644
--- a/linux-headers/linux/vfio.h
+++ b/linux-headers/linux/vfio.h
@@ -294,9 +294,10 @@ struct vfio_region_info_cap_type {
 #define VFIO_REGION_TYPE_PCI_VENDOR_MASK   (0x)
 
 /* 8086 Vendor sub-types */
-#define VFIO_REGION_SUBTYPE_INTEL_IGD_OPREGION (1)
-#define VFIO_REGION_SUBTYPE_INTEL_IGD_HOST_CFG (2)
-#define VFIO_REGION_SUBTYPE_INTEL_IGD_LPC_CFG  (3)
+#define VFIO_REGION_SUBTYPE_INTEL_IGD_OPREGION (1)
+#define VFIO_REGION_SUBTYPE_INTEL_IGD_HOST_CFG (2)
+#define VFIO_REGION_SUBTYPE_INTEL_IGD_LPC_CFG  (3)
+#define VFIO_REGION_SUBTYPE_INTEL_IGD_DEVICE_STATE (4)
 
 /**
  * VFIO_DEVICE_GET_IRQ_INFO - _IOWR(VFIO_TYPE, VFIO_BASE + 9,
-- 
2.7.4




[Qemu-devel] [RFC V2 0/4] vfio: Introduce Live migration capability to vfio_mdev device

2017-07-30 Thread Yulei Zhang
Summary

This series RFC would like to introduce the live migration capability
to vfio_mdev device. 

As currently vfio_mdev device don't support migration, we introduce 
a new vfio subtype region VFIO_REGION_SUBTYPE_INTEL_IGD_DEVICE_STATE
for Intel vGPU device, during the vfio device initialization, the mdev
device will be set to migratable if the new region exist.  

The intention to add the new region is using it for vfio_mdev device
status save and restore during the migration. The access to this region
will be trapped and forward to the vfio_mdev device driver. And we use 
the first byte in the new region to control the running state of mdev
device.

Meanwhile we add one new ioctl VFIO_IOMMU_GET_DIRTY_BITMAP to help do 
the mdev device dirty page synchronization.

So the vfio_mdev device migration sequence would be
Source VM side:
start migration
|
V
 get the cpu state change callback, write to the
 subregion's first byte to stop the mdev device
|
V
 quary the dirty page bitmap from iommu container 
 and add into qemu dirty list for synchronization
|
V
 save the deivce status into Qemufile which is 
 read from the vfio device subregion

Target VM side:
   restore the mdev device after get the
 saved status context from Qemufile
|
V
 get the cpu state change callback
 write to subregion's first byte to 
  start the mdev device to put it in 
  running status
|
V
finish migration

V1->V2:
Per Alex's suggestion:
1. use device subtype region instead of VFIO PCI fixed region.
2. remove unnecessary ioctl, use the first byte of subregion to 
   control the running state of mdev device.  
3. for dirty page synchronization, implement the interface with
   VFIOContainer instead of vfio pci device.

Yulei Zhang (4):
  vfio: introduce a new VFIO sub region for mdev device migration
support
  vfio: Add vm status change callback to stop/restart the mdev device
  vfio: Add struct vfio_vmstate_info to introduce put/get callback
funtion for vfio device status save/restore
  vifo: introduce new VFIO ioctl VFIO_IOMMU_GET_DIRTY_BITMAP

 hw/vfio/common.c  |  32 +
 hw/vfio/pci.c | 164 +-
 hw/vfio/pci.h |   1 +
 include/hw/vfio/vfio-common.h |   1 +
 linux-headers/linux/vfio.h|  26 ++-
 5 files changed, 220 insertions(+), 4 deletions(-)

-- 
2.7.4




Re: [Qemu-devel] How to make ELF headers/symbol sections available for multiboot?

2017-07-30 Thread Alexander Graf
Hi Anatol,

> Am 30.07.2017 um 23:42 schrieb Eduardo Habkost :
> 
> 
> CCing Alex, the original author of load_multiboot(), and Kevin,
> who touched multiboot code recently.
> 
> 
>> On Fri, Jul 28, 2017 at 02:28:34PM -0700, Anatol Pomozov wrote:
>> Hi
>> 
>> I am looking at x86 multiboot code and trying to add "ELF section
>> header" info feature. This will let target to

Do you have a pointer to what that feature does exactly?

>> learn more about booted
>> binary and its sections.
> 
> Are there existing OSes that use that information?
> 
> 
>> 
>> I have a draft here
>> https://github.com/anatol/qemu/commit/ad943a6eb78feee048b6bb2a1e5f49f5b686e24c
>> 
>> My understanding is that qemu multiboot loads only TEXT/BSS/DATA
>> sections. Other stuff like symbols sections and ELF headers are not
>> available for target.

Thes are not available inside the target, but they are available from the host 
(for example in -d in_asm).


Alex





[Qemu-devel] [PATCH for-2.11 2/3] qemu-iotests: remove file cleanup from bash tests

2017-07-30 Thread Jeff Cody
All files for a given test are now self-contained in a subdirectory,
and therefore the "./check" script can do all file-related cleanup
without any help.

This removes file cleanups from the bash tests.  The only cleanup left
is whatever is needed to kill any spawned processes; e.g. _cleanup_qemu.

Signed-off-by: Jeff Cody 
---
 tests/qemu-iotests/001 |  6 --
 tests/qemu-iotests/002 |  6 --
 tests/qemu-iotests/003 |  6 --
 tests/qemu-iotests/004 |  6 --
 tests/qemu-iotests/005 |  6 --
 tests/qemu-iotests/007 |  7 ---
 tests/qemu-iotests/008 |  6 --
 tests/qemu-iotests/009 |  6 --
 tests/qemu-iotests/010 |  6 --
 tests/qemu-iotests/011 |  6 --
 tests/qemu-iotests/012 |  6 --
 tests/qemu-iotests/013 |  6 --
 tests/qemu-iotests/014 |  6 --
 tests/qemu-iotests/015 |  7 ---
 tests/qemu-iotests/017 |  6 --
 tests/qemu-iotests/018 |  6 --
 tests/qemu-iotests/019 |  8 
 tests/qemu-iotests/020 |  8 
 tests/qemu-iotests/021 |  6 --
 tests/qemu-iotests/022 |  6 --
 tests/qemu-iotests/023 |  6 --
 tests/qemu-iotests/024 |  8 
 tests/qemu-iotests/025 |  6 --
 tests/qemu-iotests/026 |  7 ---
 tests/qemu-iotests/027 |  6 --
 tests/qemu-iotests/028 |  8 
 tests/qemu-iotests/029 |  7 ---
 tests/qemu-iotests/031 |  6 --
 tests/qemu-iotests/032 |  6 --
 tests/qemu-iotests/033 |  6 --
 tests/qemu-iotests/034 |  6 --
 tests/qemu-iotests/035 |  6 --
 tests/qemu-iotests/036 |  6 --
 tests/qemu-iotests/037 |  6 --
 tests/qemu-iotests/038 |  6 --
 tests/qemu-iotests/039 |  6 --
 tests/qemu-iotests/042 |  6 --
 tests/qemu-iotests/043 |  7 ---
 tests/qemu-iotests/046 |  6 --
 tests/qemu-iotests/047 |  6 --
 tests/qemu-iotests/048 |  8 
 tests/qemu-iotests/048.out |  1 -
 tests/qemu-iotests/049 |  6 --
 tests/qemu-iotests/050 |  8 
 tests/qemu-iotests/051 |  6 --
 tests/qemu-iotests/052 |  6 --
 tests/qemu-iotests/053 |  7 ---
 tests/qemu-iotests/054 |  6 --
 tests/qemu-iotests/058 |  1 -
 tests/qemu-iotests/059 |  7 ---
 tests/qemu-iotests/060 |  6 --
 tests/qemu-iotests/061 |  6 --
 tests/qemu-iotests/062 |  6 --
 tests/qemu-iotests/063 |  7 ---
 tests/qemu-iotests/064 |  6 --
 tests/qemu-iotests/066 |  6 --
 tests/qemu-iotests/068 |  6 --
 tests/qemu-iotests/069 |  6 --
 tests/qemu-iotests/070 |  6 --
 tests/qemu-iotests/071 |  6 --
 tests/qemu-iotests/072 |  6 --
 tests/qemu-iotests/073 |  6 --
 tests/qemu-iotests/074 |  9 -
 tests/qemu-iotests/074.out |  1 -
 tests/qemu-iotests/075 |  6 --
 tests/qemu-iotests/076 |  6 --
 tests/qemu-iotests/077 |  6 --
 tests/qemu-iotests/078 |  6 --
 tests/qemu-iotests/079 |  6 --
 tests/qemu-iotests/080 |  7 ---
 tests/qemu-iotests/081 |  8 
 tests/qemu-iotests/082 |  6 --
 tests/qemu-iotests/084 |  6 --
 tests/qemu-iotests/085 | 13 +
 tests/qemu-iotests/086 |  6 --
 tests/qemu-iotests/088 |  7 ---
 tests/qemu-iotests/089 |  6 --
 tests/qemu-iotests/090 |  6 --
 tests/qemu-iotests/091 |  8 
 tests/qemu-iotests/092 |  7 ---
 tests/qemu-iotests/094 |  9 +
 tests/qemu-iotests/095 |  8 +---
 tests/qemu-iotests/097 |  7 ---
 tests/qemu-iotests/098 |  7 ---
 tests/qemu-iotests/099 |  6 --
 tests/qemu-iotests/101 |  6 --
 tests/qemu-iotests/102 |  7 +--
 tests/qemu-iotests/103 |  6 --
 tests/qemu-iotests/104 |  2 --
 tests/qemu-iotests/105 |  6 --
 tests/qemu-iotests/106 |  6 --
 tests/qemu-iotests/107 |  6 --
 tests/qemu-iotests/108 |  6 --
 tests/qemu-iotests/109 |  8 +---
 tests/qemu-iotests/110 |  6 --
 tests/qemu-iotests/111 |  6 --
 tests/qemu-iotests/112 |  6 --
 tests/qemu-iotests/113 |  6 --
 tests/qemu-iotests/114 |  6 --
 tests/qemu-iotests/115 |  6 --
 tests/qemu-iotests/116 |  6 --
 tests/qemu-iotests/117 |  7 +--
 tests/qemu-iotests/119 |  6 --
 tests/qemu-iotests/120 |  6 --
 tests/qemu-iotests/121 |  6 --
 tests/qemu-iotests/122 |  7 ---
 tests/qemu-iotests/123 |  7 ---
 tests/qemu-iotests/125 |  6 --
 tests/qemu-iotests/130 |  7 +--
 tests/qemu-iotests/131 |  6 --
 tests/qemu-iotests/133 |  6 --
 tests/qemu-iotests/134 |  6 --
 tests/qemu-iotests/135 |  6 --
 tests/qemu-iotests/137 |  6 --
 tests/qemu-iotests/138 |  6 --
 tests/qemu-iotests/140 |  8 +---

[Qemu-devel] [PATCH for-2.11 3/3] qemu-iotests: add option to save temp files on error

2017-07-30 Thread Jeff Cody
Now that ./check takes care of cleaning up after each tests, it
can also selectively not clean up.  Add option to leave all output from
tests intact if that test encountered an error.

Note: this currently only works for bash tests, as the python tests
still clean up after themselves manually.

Signed-off-by: Jeff Cody 
---
 tests/qemu-iotests/check  | 5 -
 tests/qemu-iotests/common | 6 ++
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check
index 20cb93d..48d206c 100755
--- a/tests/qemu-iotests/check
+++ b/tests/qemu-iotests/check
@@ -381,7 +381,10 @@ do
 fi
 fi
 
-rm -rf "$TEST_DIR_SEQ"
+if [ "$save_on_err" != "true" ] || [ "$err" != "true" ]
+then
+rm -rf "$TEST_DIR_SEQ"
+fi
 
 fi
 
diff --git a/tests/qemu-iotests/common b/tests/qemu-iotests/common
index d34c11c..d08b233 100644
--- a/tests/qemu-iotests/common
+++ b/tests/qemu-iotests/common
@@ -42,6 +42,7 @@ expunge=true
 have_test_arg=false
 randomize=false
 cachemode=false
+save_on_err=false
 rm -f $tmp.list $tmp.tmp $tmp.sed
 
 export IMGFMT=raw
@@ -172,6 +173,7 @@ other options
 -T  output timestamps
 -r  randomize test order
 -c mode cache mode
+-s  save test scratch directory on test failure
 
 testlist options
 -g group[,group...]include tests from these groups
@@ -349,6 +351,10 @@ testlist options
 xgroup=true
 xpand=false
 ;;
+-s)
+save_on_err=true
+xpand=false
+;;
 '[0-9][0-9][0-9] [0-9][0-9][0-9][0-9]')
 echo "No tests?"
 status=1
-- 
2.9.4




[Qemu-devel] [PATCH for-2.11 0/3] qemu-iotests: place output in unique dir

2017-07-30 Thread Jeff Cody
This series does 2 things:

1.) Sets TEST_DIR to a unique subdirectory for each test
2.) Has './check' be responsible for removing temporary files
3.) Add option to './check' to retain temporary files in case of error

A caveat is #3 is working currently only for Bash tests, as the Python tests
still cleanup up after themselves.


Jeff Cody (3):
  qemu-iotests: set TEST_DIR to a unique dir for each test
  qemu-iotests: remove file cleanup from bash tests
  qemu-iotests: add option to save temp files on error

 tests/qemu-iotests/001 |  6 --
 tests/qemu-iotests/002 |  6 --
 tests/qemu-iotests/003 |  6 --
 tests/qemu-iotests/004 |  6 --
 tests/qemu-iotests/005 |  6 --
 tests/qemu-iotests/007 |  7 ---
 tests/qemu-iotests/008 |  6 --
 tests/qemu-iotests/009 |  6 --
 tests/qemu-iotests/010 |  6 --
 tests/qemu-iotests/011 |  6 --
 tests/qemu-iotests/012 |  6 --
 tests/qemu-iotests/013 |  6 --
 tests/qemu-iotests/014 |  6 --
 tests/qemu-iotests/015 |  7 ---
 tests/qemu-iotests/017 |  6 --
 tests/qemu-iotests/018 |  6 --
 tests/qemu-iotests/019 |  8 
 tests/qemu-iotests/020 |  8 
 tests/qemu-iotests/021 |  6 --
 tests/qemu-iotests/022 |  6 --
 tests/qemu-iotests/023 |  6 --
 tests/qemu-iotests/024 |  8 
 tests/qemu-iotests/025 |  6 --
 tests/qemu-iotests/026 |  7 ---
 tests/qemu-iotests/027 |  6 --
 tests/qemu-iotests/028 |  8 
 tests/qemu-iotests/029 |  7 ---
 tests/qemu-iotests/031 |  6 --
 tests/qemu-iotests/032 |  6 --
 tests/qemu-iotests/033 |  6 --
 tests/qemu-iotests/034 |  6 --
 tests/qemu-iotests/035 |  6 --
 tests/qemu-iotests/036 |  6 --
 tests/qemu-iotests/037 |  6 --
 tests/qemu-iotests/038 |  6 --
 tests/qemu-iotests/039 |  6 --
 tests/qemu-iotests/042 |  6 --
 tests/qemu-iotests/043 |  7 ---
 tests/qemu-iotests/046 |  6 --
 tests/qemu-iotests/047 |  6 --
 tests/qemu-iotests/048 |  8 
 tests/qemu-iotests/048.out |  1 -
 tests/qemu-iotests/049 |  6 --
 tests/qemu-iotests/050 |  8 
 tests/qemu-iotests/051 |  6 --
 tests/qemu-iotests/052 |  6 --
 tests/qemu-iotests/053 |  7 ---
 tests/qemu-iotests/054 |  6 --
 tests/qemu-iotests/058 |  1 -
 tests/qemu-iotests/059 |  7 ---
 tests/qemu-iotests/060 |  6 --
 tests/qemu-iotests/061 |  6 --
 tests/qemu-iotests/062 |  6 --
 tests/qemu-iotests/063 |  7 ---
 tests/qemu-iotests/064 |  6 --
 tests/qemu-iotests/066 |  6 --
 tests/qemu-iotests/068 |  6 --
 tests/qemu-iotests/069 |  6 --
 tests/qemu-iotests/070 |  6 --
 tests/qemu-iotests/071 |  6 --
 tests/qemu-iotests/072 |  6 --
 tests/qemu-iotests/073 |  6 --
 tests/qemu-iotests/074 |  9 -
 tests/qemu-iotests/074.out |  1 -
 tests/qemu-iotests/075 |  6 --
 tests/qemu-iotests/076 |  6 --
 tests/qemu-iotests/077 |  6 --
 tests/qemu-iotests/078 |  6 --
 tests/qemu-iotests/079 |  6 --
 tests/qemu-iotests/080 |  7 ---
 tests/qemu-iotests/081 |  8 
 tests/qemu-iotests/082 |  6 --
 tests/qemu-iotests/084 |  6 --
 tests/qemu-iotests/085 | 13 +
 tests/qemu-iotests/086 |  6 --
 tests/qemu-iotests/088 |  7 ---
 tests/qemu-iotests/089 |  6 --
 tests/qemu-iotests/090 |  6 --
 tests/qemu-iotests/091 |  8 
 tests/qemu-iotests/092 |  7 ---
 tests/qemu-iotests/094 |  9 +
 tests/qemu-iotests/095 |  8 +---
 tests/qemu-iotests/097 |  7 ---
 tests/qemu-iotests/098 |  7 ---
 tests/qemu-iotests/099 |  6 --
 tests/qemu-iotests/101 |  6 --
 tests/qemu-iotests/102 |  7 +--
 tests/qemu-iotests/103 |  6 --
 tests/qemu-iotests/104 |  2 --
 tests/qemu-iotests/105 |  6 --
 tests/qemu-iotests/106 |  6 --
 tests/qemu-iotests/107 |  6 --
 tests/qemu-iotests/108 |  6 --
 tests/qemu-iotests/109 |  8 +---
 tests/qemu-iotests/110 |  6 --
 tests/qemu-iotests/111 |  6 --
 tests/qemu-iotests/112 |  6 --
 tests/qemu-iotests/113 |  6 --
 tests/qemu-iotests/114 |  6 --
 tests/qemu-iotests/115 |  6 --
 tests/qemu-iotests/116 |  6 --
 tests/qemu-iotests/117 |  7 +--
 tests/qemu-iotests/119 |  6 --
 tests/qemu-iotests/120 |  6 --
 tests/qemu-iotests/121 |  6 --
 tests/qemu-iotests/122 |  7 ---
 tests/qemu-iotests/123 |  7 ---
 tests/qemu-iotests/125 |  6 --
 tests/qemu-iotests/130 |  7 +--
 tests/qemu-iotests/131 |  6 --
 tests/qemu-iotests/133 |  6 --
 tests/qemu-

[Qemu-devel] [PATCH for-2.11 1/3] qemu-iotests: set TEST_DIR to a unique dir for each test

2017-07-30 Thread Jeff Cody
Right now, all qemu-iotests output data into the same scratch directory,
and so each tests needs to be responsible for cleanup up its own files.

Have each test use 'scratch/$seq' as its temp directory, so the check
script can do simple cleanup of removing the whole temporary directory.

Signed-off-by: Jeff Cody 
---
 tests/qemu-iotests/check | 21 +
 1 file changed, 17 insertions(+), 4 deletions(-)

diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check
index 2a55ec9..20cb93d 100755
--- a/tests/qemu-iotests/check
+++ b/tests/qemu-iotests/check
@@ -254,6 +254,7 @@ seq="check"
 
 for seq in $list
 do
+TEST_DIR_SEQ=$TEST_DIR/$seq
 err=false
 printf %s "$seq"
 if [ -n "$TESTS_REMAINING_LOG" ] ; then
@@ -300,13 +301,23 @@ do
 fi
 export OUTPUT_DIR=$PWD
 if $debug; then
-(cd "$source_iotests";
+(
+cd "$source_iotests";
+export TEST_DIR=$TEST_DIR_SEQ
+. "$source_iotests/common.config"
+. "$source_iotests/common.rc"
 MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(($RANDOM % 255 + 1))} \
-$run_command -d 2>&1 | tee $tmp.out)
+$run_command -d 2>&1 | tee $tmp.out
+)
 else
-(cd "$source_iotests";
+(
+export TEST_DIR=$TEST_DIR_SEQ
+. "$source_iotests/common.config"
+. "$source_iotests/common.rc"
+ cd "$source_iotests";
 MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(($RANDOM % 255 + 1))} \
-$run_command >$tmp.out 2>&1)
+$run_command >$tmp.out 2>&1
+)
 fi
 sts=$?
 $timestamp && _timestamp
@@ -370,6 +381,8 @@ do
 fi
 fi
 
+rm -rf "$TEST_DIR_SEQ"
+
 fi
 
 # come here for each test, except when $showme is true
-- 
2.9.4




[Qemu-devel] [PULL 0/2] ppc-for-2.10 queue 20170731

2017-07-30 Thread David Gibson
The following changes since commit a588c4985eff363154d65aee8607d0a4601655f7:

  Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2017-07-28' into 
staging (2017-07-28 18:17:44 +0100)

are available in the git repository at:

  git://github.com/dgibson/qemu.git tags/ppc-for-2.10-20170731

for you to fetch changes up to fc7e0765fc385eed08c19a8823a970f4e98379b0:

  Revert "spapr: populate device tree depending on XIVE_EXPLOIT option" 
(2017-07-29 16:22:14 +1000)


ppc patch queue 2017-07-31

This has a couple of last minute bugfixes for qemu 2.10.


David Gibson (1):
  Revert "spapr: populate device tree depending on XIVE_EXPLOIT option"

Greg Kurz (1):
  spapr_drc: fix realize and unrealize

 hw/ppc/spapr.c | 10 --
 hw/ppc/spapr_drc.c | 15 ++-
 2 files changed, 10 insertions(+), 15 deletions(-)



[Qemu-devel] [PULL 2/2] Revert "spapr: populate device tree depending on XIVE_EXPLOIT option"

2017-07-30 Thread David Gibson
This reverts commit b87680427e8a3ff682f66514e99a8344e7437247.

I thought this was a harmless preliminary for XIVE enablement patches
we expect later on.  However, due to some subtle interactions between
qemu and SLOF (guest firmware) this breaks some things.  Revert it for
now, we'll work out how to fix it when the rest of the XIVE patches
are ready.

Signed-off-by: David Gibson 
---
 hw/ppc/spapr.c | 10 --
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 2a3e53d5d5..f7a19720dc 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -776,11 +776,6 @@ static int spapr_dt_cas_updates(sPAPRMachineState *spapr, 
void *fdt,
 }
 }
 
-/* /interrupt controller */
-if (!spapr_ovec_test(ov5_updates, OV5_XIVE_EXPLOIT)) {
-spapr_dt_xics(xics_max_server_number(), fdt, PHANDLE_XICP);
-}
-
 offset = fdt_path_offset(fdt, "/chosen");
 if (offset < 0) {
 offset = fdt_add_subnode(fdt, 0, "chosen");
@@ -804,7 +799,7 @@ int spapr_h_cas_compose_response(sPAPRMachineState *spapr,
 
 size -= sizeof(hdr);
 
-/* Create skeleton */
+/* Create sceleton */
 fdt_skel = g_malloc0(size);
 _FDT((fdt_create(fdt_skel, size)));
 _FDT((fdt_begin_node(fdt_skel, "")));
@@ -1077,6 +1072,9 @@ static void *spapr_build_fdt(sPAPRMachineState *spapr,
 _FDT(fdt_setprop_cell(fdt, 0, "#address-cells", 2));
 _FDT(fdt_setprop_cell(fdt, 0, "#size-cells", 2));
 
+/* /interrupt controller */
+spapr_dt_xics(xics_max_server_number(), fdt, PHANDLE_XICP);
+
 ret = spapr_populate_memory(spapr, fdt);
 if (ret < 0) {
 error_report("couldn't setup memory nodes in fdt");
-- 
2.13.3




[Qemu-devel] [PULL 1/2] spapr_drc: fix realize and unrealize

2017-07-30 Thread David Gibson
From: Greg Kurz 

If object_property_add_alias() returns an error in realize(), we should
propagate it to the caller and certainly not unref the DRC.

Same thing goes for unrealize(). Since object_property_del() is the last
call, we can even get rid of the intermediate Error *.

And finally, unrealize() should undo all registrations performed by
realize().

Signed-off-by: Greg Kurz 
Signed-off-by: David Gibson 
---
 hw/ppc/spapr_drc.c | 15 ++-
 1 file changed, 6 insertions(+), 9 deletions(-)

diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
index 15bae5c216..47d94e782a 100644
--- a/hw/ppc/spapr_drc.c
+++ b/hw/ppc/spapr_drc.c
@@ -506,11 +506,11 @@ static void realize(DeviceState *d, Error **errp)
 trace_spapr_drc_realize_child(spapr_drc_index(drc), child_name);
 object_property_add_alias(root_container, link_name,
   drc->owner, child_name, &err);
+g_free(child_name);
 if (err) {
-error_report_err(err);
-object_unref(OBJECT(drc));
+error_propagate(errp, err);
+return;
 }
-g_free(child_name);
 vmstate_register(DEVICE(drc), spapr_drc_index(drc), &vmstate_spapr_drc,
  drc);
 qemu_register_reset(drc_reset, drc);
@@ -522,16 +522,13 @@ static void unrealize(DeviceState *d, Error **errp)
 sPAPRDRConnector *drc = SPAPR_DR_CONNECTOR(d);
 Object *root_container;
 char name[256];
-Error *err = NULL;
 
 trace_spapr_drc_unrealize(spapr_drc_index(drc));
+qemu_unregister_reset(drc_reset, drc);
+vmstate_unregister(DEVICE(drc), &vmstate_spapr_drc, drc);
 root_container = container_get(object_get_root(), DRC_CONTAINER_PATH);
 snprintf(name, sizeof(name), "%x", spapr_drc_index(drc));
-object_property_del(root_container, name, &err);
-if (err) {
-error_report_err(err);
-object_unref(OBJECT(drc));
-}
+object_property_del(root_container, name, errp);
 }
 
 sPAPRDRConnector *spapr_dr_connector_new(Object *owner, const char *type,
-- 
2.13.3




Re: [Qemu-devel] [PATCH 3/3] s390x/css: generate channel path initialized CRW for channel path hotplug

2017-07-30 Thread Dong Jia Shi
* Cornelia Huck  [2017-07-27 13:59:10 +0200]:

> On Thu, 27 Jul 2017 03:54:18 +0200
> Dong Jia Shi  wrote:
> 
> > When a channel path is hot plugged into a CSS, we should generate
> > a channel path initialized CRW (channel report word). The current
> > code does not do that, instead it puts a stub function with a TODO
> > reminder there.
> > 
> > This implements the css_generate_chp_crws() function by:
> > 1. refactor the existing code.
> > 2. add an @add parameter to provide future callers with the
> >capability of generating channel path permanent error with
> >facility not initialized CRW.
> > 3. add a @hotplugged parameter, so to opt out generating initialized
> >CRWs for predefined channel paths.
> 
> I'm not 100% sure whether the logic is correct here. Let me elaborate:
> 
> The current code flow when hotplugging a device is:
> - Generate the schib.
> - Check if any of the chpids refers to a not yet existing channel path;
>   generate it if that is the case.
> - Post a crw for the subchannel.
> 
> The second step is where the current code seems to be not quite correct
> already. It is fine for coldplugged devices, but I really think we need
> to make sure that all referenced channel paths are in place before we
> hotplug a new device. It was not really relevant when we just had one
> very virtual channel path, and 3270 is experimental so it is not a
> problem in practice.
vfio-ccw hotplug could also live with the current mechanism - just
generate the chp according to its CHPIDs information. What the problem
in practice for it then? Channel path status change could be synchronize
by adding more MMIO regions and eventfd irq for vfio-ccw.

> 
> This, of course, implies we need deeper changes. We need to create the
> channel paths before the subchannel is created and refuse hotplug of a
> device if not all channel paths it needs are defined. This means we
> need some things before we can claim real channel path support:
> - Have a way to specify channel paths on the command line resp. when
>   hotplugging. This implies they need to be real objects.
> - Have a way to specify which channel paths belong to a subchannel in
>   the same context. Keep existing device types working with the current
>   method.
If we want to adopt the unified modelling for all kinds of devices, then
we require the user to define chps before define devices.

We could defaulty always have a virtio reserved chp 0 defined on each
css, so we do not need to touch the current virtio devices command line.
Defining more chps or changing chpid for virtio devices does not provide
added values.

For emulated device, we can define chpids for use. E.g.:
-device chp,cssid=fe,chpid=11 \
-device chp,cssid=fe,chpid=22 \
-chardev socket,id=terminal0,host=0.0.0.0,port=23,nowait,server,tn3270 \
-device 
x-terminal3270,chardev=terminal0,id=terminal3270_0,devno=fe.0.000a,chpids=1122

Or, I think, we could let Qemu automatically find a free chp for them.
Sine, the same as the virtio devices, defining more chps or changing
chpid for emulated devices does provide added values either. In this
case, we do not need to touch the emualted device command line too.

When defining a vfio-ccw device, since the real subchannel implicitly
indicates the chps it bound to, we grasp the CHPIDs from sysfs (or, with
my current work, we could even retrieve these information from a new
added MMIO region). In this case, defining some channel path devices
separately does not make sense to me.

After thinking quite a while, if we do want to add a real device object
for a channel path, the most intractable problem (but not the only one)
for me is to find a good way to map the real path with the virtual one.
How would we retrieve the information from the real one? We'd need the
host kernel to provide totally new interfaces for channel path
information synchronization and notification machenism. I don't think in
this case sysfs is the choice. Ioctls, vfio MMIO regions and eventfd
could be a better choice. I think, this is like we are trying to
passthru a channel path. So we'd need to have a new vfio device physical
driver (e.g. vfio-chp) to handle this...

And, if we finnaly find a way to solve the above problem, we may have
some commandline as the follows, and there is still other problems. E.g.:

lscss:
MDEV  Subchan.  PIM PAM POM  CHPIDs
--
6dfd3ec5-e8b3-4e18-a6fe-57bc9eceb920  0.0.013f  f0  f0  ff   42434445 

lschp:
CHPID  Vary  Cfg.  Type  Cmg  Shared  PCHID

0.42   1 1 1b21   0158 
0.43   1 1 1b21   0159 
0.44   1 1 1b21   01a0 
0.45   1 1 1b21   01a1

Suppose we want to pass through the above mdev ($MDEV_CCW013f), we could
have the following command line:
-device vfio-chp,sysfsdev=$MDEV_CHP42,cssid=0,chpid=42 \

Re: [Qemu-devel] [for-2.11 PATCH 18/26] spapr: create DR connectors for PHBs

2017-07-30 Thread David Gibson
On Fri, Jul 28, 2017 at 12:30:35PM +0200, Greg Kurz wrote:
> On Fri, 28 Jul 2017 13:49:25 +1000
> David Gibson  wrote:
> 
> > On Tue, Jul 25, 2017 at 08:01:50PM +0200, Greg Kurz wrote:
> > > From: Michael Roth 
> > > 
> > > Signed-off-by: Michael Roth 
> > > Reviewed-by: David Gibson 
> > > Signed-off-by: Greg Kurz   
> > 
> > 
> > 
> > > ---
> > > Changes since RFC:
> > > - rebased against ppc-for-2.10 (reset hooks registering already merged)
> > > - added new DRC type for PHB
> > > ---
> > >  hw/ppc/spapr.c |   15 +++
> > >  hw/ppc/spapr_drc.c |   17 +
> > >  include/hw/ppc/spapr_drc.h |8 
> > >  3 files changed, 40 insertions(+)
> > > 
> > > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > > index 8dc505343c0f..5950c009ab7e 100644
> > > --- a/hw/ppc/spapr.c
> > > +++ b/hw/ppc/spapr.c
> > > @@ -98,6 +98,9 @@
> > >  
> > >  #define PHANDLE_XICP0x
> > >  
> > > +/* maximum number of hotpluggable PHBs */
> > > +#define SPAPR_DRC_MAX_PHB   256  
> > 
> > I wonder if we should actually make this a machine property.
> > 
> 
> It makes sense.
> 
> Also, if all PHBs are instanciated with index != -1, we're limited to 31.
> Maybe this could be the default value for the machine property instead of
> 256 then ?

Actually, if we're binding it back to index, which has a hard limit,
then it no longer makes sense to have it as a property and we should
go back to a constant (well, it could vary by machine type version).

> > >  static ICSState *spapr_ics_create(sPAPRMachineState *spapr,
> > >const char *type_ics,
> > >int nr_irqs, Error **errp)
> > > @@ -2384,6 +2387,18 @@ static void ppc_spapr_init(MachineState *machine)
> > >  
> > >  spapr->dr_phb_enabled = smc->dr_phb_enabled;
> > >  
> > > +/* Setup hotplug / dynamic-reconfiguration connectors. top-level
> > > + * connectors (described in root DT node's "ibm,drc-types" property)
> > > + * are pre-initialized here. additional child connectors (such as
> > > + * connectors for a PHBs PCI slots) are added as needed during their
> > > + * parent's realization.
> > > + */
> > > +if (spapr->dr_phb_enabled) {
> > > +for (i = 0; i < SPAPR_DRC_MAX_PHB; i++) {
> > > +spapr_dr_connector_new(OBJECT(machine), TYPE_SPAPR_DRC_PHB, 
> > > i);
> > > +}
> > > +}
> > > +
> > >  /* Set up PCI */
> > >  spapr_pci_rtas_init();
> > >  
> > > diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
> > > index eb8024d37c54..2e1049ce61c7 100644
> > > --- a/hw/ppc/spapr_drc.c
> > > +++ b/hw/ppc/spapr_drc.c
> > > @@ -697,6 +697,15 @@ static void spapr_drc_lmb_class_init(ObjectClass *k, 
> > > void *data)
> > >  drck->release = spapr_lmb_release;
> > >  }
> > >  
> > > +static void spapr_drc_phb_class_init(ObjectClass *k, void *data)
> > > +{
> > > +sPAPRDRConnectorClass *drck = SPAPR_DR_CONNECTOR_CLASS(k);
> > > +
> > > +drck->typeshift = SPAPR_DR_CONNECTOR_TYPE_SHIFT_PHB;
> > > +drck->typename = "PHB";
> > > +drck->drc_name_prefix = "PHB ";
> > > +}
> > > +
> > >  static const TypeInfo spapr_dr_connector_info = {
> > >  .name  = TYPE_SPAPR_DR_CONNECTOR,
> > >  .parent= TYPE_DEVICE,
> > > @@ -740,6 +749,13 @@ static const TypeInfo spapr_drc_lmb_info = {
> > >  .class_init= spapr_drc_lmb_class_init,
> > >  };
> > >  
> > > +static const TypeInfo spapr_drc_phb_info = {
> > > +.name  = TYPE_SPAPR_DRC_PHB,
> > > +.parent= TYPE_SPAPR_DRC_LOGICAL,  
> > 
> > I thought PHB DRCs were physical..
> > 
> 
> My understanding is that only PCI IOAs need a physical DRC.
> 
> From LoPAPR v1.1 (March 24, 2016):
> 
> 13.7 Logical Resource Dynamic Reconfiguration (LRDR)
> 
> The Logical Resource Dynamic Reconfiguration option allows a platform to make 
> available and recover platform re-
> sources such as CPUs, Memory Regions, Processor Host Bridges, and I/O slots 
> to/from its operating OS image(s).
> 
> ...
> 
> The device tree contains logical resource DR connectors for the maximum 
> number of resources that the platform can
> allocate to the specific OS. In some cases such as for processors and PHBs...
> 
> and
> 
> Table 240. Currently Defined DR Connector Types
> 
> | PHB | Logical PCI Host Bridge |

Ah, my mistake.

> 
> > > +.instance_size = sizeof(sPAPRDRConnector),
> > > +.class_init= spapr_drc_phb_class_init,
> > > +};
> > > +
> > >  /* helper functions for external users */
> > >  
> > >  sPAPRDRConnector *spapr_drc_by_index(uint32_t index)
> > > @@ -1179,6 +1195,7 @@ static void spapr_drc_register_types(void)
> > >  type_register_static(&spapr_drc_cpu_info);
> > >  type_register_static(&spapr_drc_pci_info);
> > >  type_register_static(&spapr_drc_lmb_info);
> > > +type_register_static(&spapr_drc_phb_info);
> > >  
> > >  spapr_rtas_register(RTAS_SET_INDICATOR, "set-indi

Re: [Qemu-devel] [PATCH 3/3] s390x/css: generate channel path initialized CRW for channel path hotplug

2017-07-30 Thread Dong Jia Shi
* Cornelia Huck  [2017-07-28 14:58:19 +0200]:

[...]

> > 
> > If I understand you correctly it ain't possible to handle these
> > in the host (and let the guest a simple 'non-real' virtual
> > channel path whose reliability depends on what the host does),
> > or?
> 
> It is possible. Mapping to a virtual channel path or not is basically a
> design decision (IIRC, z/VM supports both).
> 
> Mapping everything to a virtual chpid basically concentrates all
> path-related handling in the hypervisor. This allows for a dumb guest
> OS, but can make errors really hard to debug from the guest side.
I understood this.

> 
> Exposing real channel paths to the guest means that the guest OS needs
> to be able to deal with path-related things, but OTOH it has more
> control. As I don't think we'll ever want to support a guest OS that
> does not also run under LPAR, I'd prefer that way.
> 
My poor English... Sorry, I don't undersatnd the last sentence...

[...]

-- 
Dong Jia Shi




Re: [Qemu-devel] [PATCH v8 3/3] migration: add bitmap for received page

2017-07-30 Thread Peter Xu
On Fri, Jul 28, 2017 at 06:29:20PM +0300, Alexey Perevalov wrote:
> On 07/28/2017 10:06 AM, Alexey Perevalov wrote:
> >On 07/28/2017 09:57 AM, Peter Xu wrote:
> >>On Fri, Jul 28, 2017 at 09:43:28AM +0300, Alexey Perevalov wrote:
> >>>On 07/28/2017 07:27 AM, Peter Xu wrote:
> On Thu, Jul 27, 2017 at 10:27:41AM +0300, Alexey Perevalov wrote:
> >On 07/27/2017 05:35 AM, Peter Xu wrote:
> >>On Wed, Jul 26, 2017 at 06:24:11PM +0300, Alexey Perevalov wrote:
> >>>On 07/26/2017 11:43 AM, Peter Xu wrote:
> On Wed, Jul 26, 2017 at 11:07:17AM +0300, Alexey Perevalov wrote:
> >On 07/26/2017 04:49 AM, Peter Xu wrote:
> >>On Thu, Jul 20, 2017 at 09:52:34AM +0300, Alexey
> >>Perevalov wrote:
> >>>This patch adds ability to track down already received
> >>>pages, it's necessary for calculation vCPU block time in
> >>>postcopy migration feature, maybe for restore after
> >>>postcopy migration failure.
> >>>Also it's necessary to solve shared memory issue in
> >>>postcopy livemigration. Information about received pages
> >>>will be transferred to the software virtual bridge
> >>>(e.g. OVS-VSWITCHD), to avoid fallocate (unmap) for
> >>>already received pages. fallocate syscall is required for
> >>>remmaped shared memory, due to remmaping itself blocks
> >>>ioctl(UFFDIO_COPY, ioctl in this case will end with EEXIT
> >>>error (struct page is exists after remmap).
> >>>
> >>>Bitmap is placed into RAMBlock as another postcopy/precopy
> >>>related bitmaps.
> >>>
> >>>Reviewed-by: Peter Xu 
> >>>Signed-off-by: Alexey Perevalov 
> >>>---
> >>[...]
> >>
> >>>  static int qemu_ufd_copy_ioctl(int userfault_fd,
> >>>void *host_addr,
> >>>-void *from_addr, uint64_t pagesize)
> >>>+   void *from_addr,
> >>>uint64_t pagesize, RAMBlock *rb)
> >>>  {
> >>>+int ret;
> >>>  if (from_addr) {
> >>>  struct uffdio_copy copy_struct;
> >>>  copy_struct.dst = (uint64_t)(uintptr_t)host_addr;
> >>>  copy_struct.src = (uint64_t)(uintptr_t)from_addr;
> >>>  copy_struct.len = pagesize;
> >>>  copy_struct.mode = 0;
> >>>-return ioctl(userfault_fd, UFFDIO_COPY, ©_struct);
> >>>+ret = ioctl(userfault_fd, UFFDIO_COPY, ©_struct);
> >>>  } else {
> >>>  struct uffdio_zeropage zero_struct;
> >>>  zero_struct.range.start =
> >>>(uint64_t)(uintptr_t)host_addr;
> >>>  zero_struct.range.len = pagesize;
> >>>  zero_struct.mode = 0;
> >>>-return ioctl(userfault_fd, UFFDIO_ZEROPAGE,
> >>>&zero_struct);
> >>>+ret = ioctl(userfault_fd, UFFDIO_ZEROPAGE,
> >>>&zero_struct);
> >>>+}
> >>>+if (!ret) {
> >>>+ramblock_recv_bitmap_set(host_addr, rb);
> >>Wait...
> >>
> >>Now we are using 4k-page/bit bitmap, do we need to take
> >>care of the
> >>huge pages here?  Looks like we are only setting the
> >>first bit of it
> >>if it is a huge page?
> >First version was per ramblock page size, IOW bitmap was
> >smaller in
> >case of hugepages.
> Yes, but this is not the first version any more. :)
> 
> This patch is using:
> 
>    bitmap_new(rb->max_length >> TARGET_PAGE_BITS);
> 
> to allocate bitmap, so it is using small pages always for bitmap,
> right? (I should not really say "4k" pages, here I think the
> size is
> host page size, which is the thing returned from getpagesize()).
> 
> >You mentioned that TARGET_PAGE_SIZE is reasonable for
> >precopy case,
> >in "Re: [Qemu-devel] [PATCH v1 2/2] migration: add bitmap
> >for copied page"
> >I though TARGET_PAGE_SIZE as transmition unit, is using in
> >precopy even
> >hugepage case.
> >But it's not so logically, page being marked as dirty,
> >should be sent as a
> >whole page.
> Sorry if I misunderstood, but I didn't see anything wrong - we are
> sending pages in small pages, but when postcopy is there, we do
> UFFDIO_COPY in huge page, so everything is fine?
> >>>I think yes, we chose TARGET_PAGE_SIZE because of wider
> >>>use case ranges.
> >>So... are you going to post another version? IIUC we just need
> >>to use
> >>a bitmap_set() to replace the ramblock_recv_bitmap_set(), while set
> >>the size with "pagesize / TARGET_PAGE_SIZE"?
> > From my point of view TARGET_PAGE_SIZE/TARGET_PAGE_BITS it's a
> >platform
> >specific
> >
> 

Re: [Qemu-devel] [PATCH] 9pfs: include for XATTR_SIZE_MAX

2017-07-30 Thread Kamil Rytarowski
On 30.07.2017 20:23, Peter Maydell wrote:
> On 30 July 2017 at 17:51, Kamil Rytarowski  wrote:
>> On 29.07.2017 21:34, Peter Maydell wrote:
>>> On 29 July 2017 at 14:50, Patrick Steinhardt  wrote:
 On Fri, Jul 28, 2017 at 02:20:49PM -0300, Philippe Mathieu-Daudé wrote:
> This is likely to break on BSD, but now than patchew has a NetBSD job
> you can trigger a build RESENDing this patch.
>>>
>>
>> I just checked patchew, and there is FreeBSD job. How far are we from
>> adding more BSDs?
> 
> I now test OpenBSD and NetBSD as well in my pre-merge
> test setup. Patchew could add them as well if desired.
> (vm setup instructions at http://wiki.qemu.org/Hosts/BSD)
> 

Please do.

> (I haven't bothered to send a patch marking OpenBSD
> as 'supported' since we've had zero contact from
> anybody in the OpenBSD community AFAIK.)
> 

There is one maintainer in OpenBSD ports Brad Smith, but he's not an
OpenBSD developer as far as I can tell.

Adding him to CC.

> thanks
> -- PMM
> 




signature.asc
Description: OpenPGP digital signature


Re: [Qemu-devel] How to make ELF headers/symbol sections available for multiboot?

2017-07-30 Thread Eduardo Habkost

CCing Alex, the original author of load_multiboot(), and Kevin,
who touched multiboot code recently.


On Fri, Jul 28, 2017 at 02:28:34PM -0700, Anatol Pomozov wrote:
> Hi
> 
> I am looking at x86 multiboot code and trying to add "ELF section
> header" info feature. This will let target to learn more about booted
> binary and its sections.

Are there existing OSes that use that information?


> 
> I have a draft here
> https://github.com/anatol/qemu/commit/ad943a6eb78feee048b6bb2a1e5f49f5b686e24c
> 
> My understanding is that qemu multiboot loads only TEXT/BSS/DATA
> sections. Other stuff like symbols sections and ELF headers are not
> available for target.
> 
> So I need to perform 2 things:
> 
>  - Load ELF section headers into target's memory. I did by appending
> additional space to mbs.mb_buf and copying header data. Is it the best
> way to do?
> 
>  - Next I need to load other ELF sections such as symbols (e.g.
> .shstrtab) that store section names. What is the best way to do in
> multiboo.c code? Would it make sense to load all ELF sections?
> 
> Thanks in advance.
> 

-- 
Eduardo



Re: [Qemu-devel] [PATCH v5 3/6] qemu.py: cleanup message on negative exit code

2017-07-30 Thread Eduardo Habkost
On Thu, Jul 27, 2017 at 10:21:22AM +0200, Amador Pahim wrote:
> On Tue, Jul 25, 2017 at 9:51 PM, Eduardo Habkost  wrote:
> > On Tue, Jul 25, 2017 at 07:10:11PM +0200, Amador Pahim wrote:
> >> The message contains the self._args, which has only part of the
> >> options used in the qemu command line and is not representative
> >> enough to figure out what happened to the process.
> >>
> >> This patch drops the self._args part of the message.
> >>
> >> Signed-off-by: Amador Pahim 
> >
> > I actually think it is a very useful debugging message as is,
> > because the command-line arguments are often all we need to
> > reproduce a QEMU crash.
> 
> The message currently contains only part of the args, not all
> (base_args are not included). Let's include the full command then.
> 
> >
> > That said, sys.stderr.write doesn't belong to the QEMUMachine
> > code, as callers should decide if/when/how/where to print
> > information about a QEMU crash.
> >
> > I think a QEMUCrashed exception class would be the best way to
> > report that to callers.  Including the full QEMU command-line on
> > the exception __str__ method would make it helpful when debugging
> > crashes: existing code that doesn't catch launch() exceptions
> > will crash with a more helpful stack trace, and code that already
> > catches exceptions is probably going to print exception info
> > somewhere.
> 
> I agree using sys.stderr.write should be avoided, but I'm not
> convinced this message should raise an exception. [...]

No problem, we can discuss later when/how to raise exceptions to
indicate specific error cases.

We could make the log message conditional on self._debug by now,
but I don't think it will be a problem if we keep it
unconditional (as QEMU crashes are not supposed to happen under
normal circumstances).

> [...] I think it's time to
> improve the logging capabilities here.

What about using the Python logging module?

-- 
Eduardo



[Qemu-devel] [PATCH v2 2/2] Add more function keys to cocoa.m

2017-07-30 Thread Programmingkid
Enable the Cocoa front-end to be able to use function keys F16 to F20.

Signed-off-by: John Arbuckle 
---
 ui/cocoa.m | 5 +
 1 file changed, 5 insertions(+)

diff --git a/ui/cocoa.m b/ui/cocoa.m
index 93e56d0..edc7a65 100644
--- a/ui/cocoa.m
+++ b/ui/cocoa.m
@@ -225,6 +225,11 @@ const int mac_to_qkeycode_map[] = {
 [kVK_F13] = Q_KEY_CODE_PRINT,
 [kVK_F14] = Q_KEY_CODE_SCROLL_LOCK,
 [kVK_F15] = Q_KEY_CODE_PAUSE,
+[kVK_F16] = Q_KEY_CODE_F16,
+[kVK_F17] = Q_KEY_CODE_F17,
+[kVK_F18] = Q_KEY_CODE_F18,
+[kVK_F19] = Q_KEY_CODE_F19,
+[kVK_F20] = Q_KEY_CODE_F20

 /*
  * The eject and volume keys can't be used here because they are handled at
-- 
2.7.2





[Qemu-devel] [PATCH v2 1/2] Add more function keys to QEMU

2017-07-30 Thread Programmingkid
There are now keyboards that have 19 function keys. This patch extends QEMU so 
these function keys can be used.

Signed-off-by: John Arbuckle 
---
 qapi-schema.json  | 12 +++-
 ui/input-keymap.c |  9 +
 2 files changed, 20 insertions(+), 1 deletion(-)

diff --git a/qapi-schema.json b/qapi-schema.json
index c96f0a2..f1c989b 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -4862,6 +4862,15 @@
 # @ac_refresh: since 2.10
 # @ac_bookmarks: since 2.10
 # altgr, altgr_r: dropped in 2.10
+# @f16: since 2.11
+# @f17: since 2.11
+# @f18: since 2.11
+# @f19: since 2.11
+# @f20: since 2.11
+# @f21: since 2.11
+# @f22: since 2.11
+# @f23: since 2.11
+# @f24: since 2.11
 #
 # Since: 1.3.0
 #
@@ -4888,7 +4897,8 @@
 'audionext', 'audioprev', 'audiostop', 'audioplay', 'audiomute',
 'volumeup', 'volumedown', 'mediaselect',
 'mail', 'calculator', 'computer',
-'ac_home', 'ac_back', 'ac_forward', 'ac_refresh', 'ac_bookmarks' ] 
}
+'ac_home', 'ac_back', 'ac_forward', 'ac_refresh', 'ac_bookmarks',
+'f16', 'f17', 'f18', 'f19', 'f20', 'f21', 'f22', 'f23', 'f24'] }
 
 ##
 # @KeyValue:
diff --git a/ui/input-keymap.c b/ui/input-keymap.c
index cf979c2..c0413e1 100644
--- a/ui/input-keymap.c
+++ b/ui/input-keymap.c
@@ -251,6 +251,15 @@ static const int qcode_to_number[] = {
 
 [Q_KEY_CODE_F11] = 0x57,
 [Q_KEY_CODE_F12] = 0x58,
+[Q_KEY_CODE_F16] = 0x59,
+[Q_KEY_CODE_F17] = 0x5a,
+[Q_KEY_CODE_F18] = 0x5b,
+[Q_KEY_CODE_F19] = 0x5c,
+[Q_KEY_CODE_F20] = 0x5d,
+[Q_KEY_CODE_F21] = 0x5e,
+[Q_KEY_CODE_F22] = 0x5f,
+[Q_KEY_CODE_F23] = 0x60,
+[Q_KEY_CODE_F24] = 0x61,
 
 [Q_KEY_CODE_PRINT] = 0xb7,
 
-- 
2.7.2





[Qemu-devel] [PATCH v2 0/2] Add more function keys support

2017-07-30 Thread Programmingkid
QEMU can only handle up to function key F15. These patches extend QEMU to F24.

John Arbuckle (2):
  Add more function keys to QEMU
  Add more function keys to cocoa.m

 qapi-schema.json  | 13 -
 ui/cocoa.m|  5 +
 ui/input-keymap.c |  9 +
 3 files changed, 26 insertions(+), 1 deletion(-)

-- 
2.7.2





Re: [Qemu-devel] [PATCH] 9pfs: include for XATTR_SIZE_MAX

2017-07-30 Thread Peter Maydell
On 30 July 2017 at 17:51, Kamil Rytarowski  wrote:
> On 29.07.2017 21:34, Peter Maydell wrote:
>> On 29 July 2017 at 14:50, Patrick Steinhardt  wrote:
>>> On Fri, Jul 28, 2017 at 02:20:49PM -0300, Philippe Mathieu-Daudé wrote:
 This is likely to break on BSD, but now than patchew has a NetBSD job
 you can trigger a build RESENDing this patch.
>>
>
> I just checked patchew, and there is FreeBSD job. How far are we from
> adding more BSDs?

I now test OpenBSD and NetBSD as well in my pre-merge
test setup. Patchew could add them as well if desired.
(vm setup instructions at http://wiki.qemu.org/Hosts/BSD)

(I haven't bothered to send a patch marking OpenBSD
as 'supported' since we've had zero contact from
anybody in the OpenBSD community AFAIK.)

thanks
-- PMM



Re: [Qemu-devel] [PATCH] 9pfs: include for XATTR_SIZE_MAX

2017-07-30 Thread Kamil Rytarowski
On 29.07.2017 21:34, Peter Maydell wrote:
> On 29 July 2017 at 14:50, Patrick Steinhardt  wrote:
>> On Fri, Jul 28, 2017 at 02:20:49PM -0300, Philippe Mathieu-Daudé wrote:
>>> This is likely to break on BSD, but now than patchew has a NetBSD job
>>> you can trigger a build RESENDing this patch.
> 

I just checked patchew, and there is FreeBSD job. How far are we from
adding more BSDs?



signature.asc
Description: OpenPGP digital signature


Re: [Qemu-devel] [PATCH v12 5/8] virtio-balloon: VIRTIO_BALLOON_F_SG

2017-07-30 Thread Michael S. Tsirkin
On Sun, Jul 30, 2017 at 07:18:33PM +0300, Michael S. Tsirkin wrote:
> On Sun, Jul 30, 2017 at 05:59:17AM +, Wang, Wei W wrote:
> > On Sunday, July 30, 2017 12:23 PM, Michael S. Tsirkin wrote:
> > > On Sat, Jul 29, 2017 at 08:47:08PM +0800, Wei Wang wrote:
> > > > On 07/29/2017 07:08 AM, Michael S. Tsirkin wrote:
> > > > > On Thu, Jul 27, 2017 at 10:50:11AM +0800, Wei Wang wrote:
> > > > > > > > > OK I thought this over. While we might need these new APIs
> > > > > > > > > in the future, I think that at the moment, there's a way to
> > > > > > > > > implement this feature that is significantly simpler. Just
> > > > > > > > > add each s/g as a separate input buffer.
> > > > > > > > Should it be an output buffer?
> > > > > > > Hypervisor overwrites these pages with zeroes. Therefore it is
> > > > > > > writeable by device: DMA_FROM_DEVICE.
> > > > > > Why would the hypervisor need to zero the buffer?
> > > > > The page is supplied to hypervisor and can lose the value that is
> > > > > there.  That is the definition of writeable by device.
> > > >
> > > > I think for the free pages, it should be clear that they will be added
> > > > as output buffer to the device, because (as we discussed) they are
> > > > just hints, and some of them may be used by the guest after the report_ 
> > > > API is
> > > invoked.
> > > > The device/hypervisor should not use or discard them.
> > > 
> > > Discarding contents is exactly what you propose doing if migration is 
> > > going on,
> > > isn't it?
> > 
> > That's actually a different concept. Please let me explain it with this 
> > example:
> > 
> > The hypervisor receives the hint saying the guest PageX is a free page, but 
> > as we know, 
> > after that report_ API exits, the guest kernel may take PageX to use, so 
> > PageX is not free
> > page any more. At this time, if the hypervisor writes to the page, that 
> > would crash the guest.
> > So, I think the cornerstone of this work is that the hypervisor should not 
> > touch the
> > reported pages.
> > 
> > Best,
> > Wei
> 
> That's a hypervisor implementation detail. From guest point of view,
> discarding contents can not be distinguished from writing old contents.
> 

Besides, ignoring the free page tricks, consider regular ballooning.
We map page with DONTNEED then back with WILLNEED. Result is
getting a zero page. So at least one of deflate/inflate should be input.
I'd say both for symmetry.

-- 
MST



Re: [Qemu-devel] [PATCH v12 5/8] virtio-balloon: VIRTIO_BALLOON_F_SG

2017-07-30 Thread Michael S. Tsirkin
On Sun, Jul 30, 2017 at 05:59:17AM +, Wang, Wei W wrote:
> On Sunday, July 30, 2017 12:23 PM, Michael S. Tsirkin wrote:
> > On Sat, Jul 29, 2017 at 08:47:08PM +0800, Wei Wang wrote:
> > > On 07/29/2017 07:08 AM, Michael S. Tsirkin wrote:
> > > > On Thu, Jul 27, 2017 at 10:50:11AM +0800, Wei Wang wrote:
> > > > > > > > OK I thought this over. While we might need these new APIs
> > > > > > > > in the future, I think that at the moment, there's a way to
> > > > > > > > implement this feature that is significantly simpler. Just
> > > > > > > > add each s/g as a separate input buffer.
> > > > > > > Should it be an output buffer?
> > > > > > Hypervisor overwrites these pages with zeroes. Therefore it is
> > > > > > writeable by device: DMA_FROM_DEVICE.
> > > > > Why would the hypervisor need to zero the buffer?
> > > > The page is supplied to hypervisor and can lose the value that is
> > > > there.  That is the definition of writeable by device.
> > >
> > > I think for the free pages, it should be clear that they will be added
> > > as output buffer to the device, because (as we discussed) they are
> > > just hints, and some of them may be used by the guest after the report_ 
> > > API is
> > invoked.
> > > The device/hypervisor should not use or discard them.
> > 
> > Discarding contents is exactly what you propose doing if migration is going 
> > on,
> > isn't it?
> 
> That's actually a different concept. Please let me explain it with this 
> example:
> 
> The hypervisor receives the hint saying the guest PageX is a free page, but 
> as we know, 
> after that report_ API exits, the guest kernel may take PageX to use, so 
> PageX is not free
> page any more. At this time, if the hypervisor writes to the page, that would 
> crash the guest.
> So, I think the cornerstone of this work is that the hypervisor should not 
> touch the
> reported pages.
> 
> Best,
> Wei

That's a hypervisor implementation detail. From guest point of view,
discarding contents can not be distinguished from writing old contents.




Re: [Qemu-devel] [PATCH v8 0/5] hypertrace: Lightweight guest-to-QEMU trace channel

2017-07-30 Thread no-reply
Hi,

This series seems to have some coding style problems. See output below for
more information:

Subject: [Qemu-devel] [PATCH v8 0/5] hypertrace: Lightweight guest-to-QEMU 
trace channel
Message-id: 150142369849.12995.11229612194223213120.st...@frigg.lan
Type: series

=== TEST SCRIPT BEGIN ===
#!/bin/bash

BASE=base
n=1
total=$(git log --oneline $BASE.. | wc -l)
failed=0

git config --local diff.renamelimit 0
git config --local diff.renames True

commits="$(git log --format=%H --reverse $BASE..)"
for c in $commits; do
echo "Checking PATCH $n/$total: $(git log -n 1 --format=%s $c)..."
if ! git show $c --format=email | ./scripts/checkpatch.pl --mailback -; then
failed=1
echo
fi
n=$((n+1))
done

exit $failed
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
b727925815 hypertrace: Add guest-side user-level library
bb83766b6c hypertrace: [softmmu] Add QEMU-side proxy to "guest_hypertrace" event
b1e07ad5aa hypertrace: [*-user] Add QEMU-side proxy to "guest_hypertrace" event
add0b54fc3 hypertrace: Add tracing event "guest_hypertrace"
cecce1a6d5 hypertrace: Add documentation

=== OUTPUT BEGIN ===
Checking PATCH 1/5: hypertrace: Add documentation...
Checking PATCH 2/5: hypertrace: Add tracing event "guest_hypertrace"...
Checking PATCH 3/5: hypertrace: [*-user] Add QEMU-side proxy to 
"guest_hypertrace" event...
Checking PATCH 4/5: hypertrace: [softmmu] Add QEMU-side proxy to 
"guest_hypertrace" event...
Checking PATCH 5/5: hypertrace: Add guest-side user-level library...
WARNING: architecture specific defines should be avoided
#171: FILE: hypertrace/guest/common.c:84:
+#if !defined(CONFIG_USER_ONLY) && defined(__linux__)

ERROR: consider using qemu_strtoull in preference to strtoull
#191: FILE: hypertrace/guest/common.c:104:
+uint64_t vv = strtoull(v, &res, 16);

total: 1 errors, 1 warnings, 432 lines checked

Your patch has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

=== OUTPUT END ===

Test command exited with code: 1


---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-de...@freelists.org

[Qemu-devel] [PATCH v8 5/5] hypertrace: Add guest-side user-level library

2017-07-30 Thread Lluís Vilanova
Provides guest library "libqemu-hypertrace-guest.a" to abstract access
to the hypertrace channel.

Signed-off-by: Lluís Vilanova 
---
 Makefile   |5 +
 configure  |2 
 hypertrace/guest/Makefile  |   30 
 hypertrace/guest/common.c  |  301 
 hypertrace/guest/qemu-hypertrace.h |   80 ++
 5 files changed, 418 insertions(+)
 create mode 100644 hypertrace/guest/Makefile
 create mode 100644 hypertrace/guest/common.c
 create mode 100644 hypertrace/guest/qemu-hypertrace.h

diff --git a/Makefile b/Makefile
index f08f498f79..523885a0a5 100644
--- a/Makefile
+++ b/Makefile
@@ -602,8 +602,13 @@ ifneq (,$(findstring qemu-ga,$(TOOLS)))
 endif
 endif
 
+install-hypertrace:
+   $(INSTALL_DIR) "$(DESTDIR)$(includedir)"
+   $(INSTALL_DATA) "$(SRC_PATH)/hypertrace/guest/qemu-hypertrace.h" 
"$(DESTDIR)$(includedir)/"
+
 
 install: all $(if $(BUILD_DOCS),install-doc) install-datadir 
install-localstatedir
+install: install-hypertrace
 ifneq ($(TOOLS),)
$(call install-prog,$(subst 
qemu-ga,qemu-ga$(EXESUF),$(TOOLS)),$(DESTDIR)$(bindir))
 endif
diff --git a/configure b/configure
index e6f752c242..9c7358d0b4 100755
--- a/configure
+++ b/configure
@@ -6286,6 +6286,8 @@ if [ "$TARGET_BASE_ARCH" = "" ]; then
 fi
 
 symlink "$source_path/Makefile.target" "$target_dir/Makefile"
+mkdir -p $target_dir/hypertrace/guest
+symlink $source_path/hypertrace/guest/Makefile 
$target_dir/hypertrace/guest/Makefile
 
 upper() {
 echo "$@"| LC_ALL=C tr '[a-z]' '[A-Z]'
diff --git a/hypertrace/guest/Makefile b/hypertrace/guest/Makefile
new file mode 100644
index 00..cbc956bde1
--- /dev/null
+++ b/hypertrace/guest/Makefile
@@ -0,0 +1,30 @@
+include ../../../config-host.mak
+include ../../config-target.mak
+include $(SRC_PATH)/rules.mak
+
+vpath % $(SRC_PATH)/hypertrace/guest
+
+# do not use QEMU's per-host cflags when building guest code
+QEMU_CFLAGS  = -Werror -Wall
+
+QEMU_CFLAGS += $(GLIB_CFLAGS)
+QEMU_CFLAGS += -I$(SRC_PATH)/include
+QEMU_CFLAGS += -I../../../linux-headers
+QEMU_CFLAGS += -I../../../
+QEMU_CFLAGS += -I../../
+
+ifdef CONFIG_SOFTMMU
+QEMU_CFLAGS += -DNEED_CPU_H
+QEMU_CFLAGS += -I$(SRC_PATH)/target-$(TARGET_BASE_ARCH)
+endif
+
+QEMU_CFLAGS += -g -O2
+
+obj-y = common.o
+
+libqemu-hypertrace-guest.a: $(obj-y)
+
+all: libqemu-hypertrace-guest.a
+
+clean:
+   rm -f $(obj-y) libqemu-hypertrace-guest.a
diff --git a/hypertrace/guest/common.c b/hypertrace/guest/common.c
new file mode 100644
index 00..96ae4464bc
--- /dev/null
+++ b/hypertrace/guest/common.c
@@ -0,0 +1,301 @@
+/*
+ * Guest-side management of hypertrace.
+ *
+ * Copyright (C) 2016-2017 Lluís Vilanova 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu-hypertrace.h"
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "config-host.h"
+#include "config-target.h"
+#if defined(CONFIG_SOFTMMU)
+#include "qemu/osdep.h"
+#include "hw/pci/pci.h"
+#endif
+#include "hypertrace/common.h"
+
+static pthread_mutex_t init_mutex = PTHREAD_MUTEX_INITIALIZER;
+
+static char *config_path;
+static int config_fd = -1;
+static uint64_t *config_addr;
+static struct hypertrace_config *config;
+
+static char *data_path;
+static int data_fd = -1;
+static uint64_t *data_addr;
+
+static char *control_path;
+static int control_fd = -1;
+#if defined(CONFIG_USER_ONLY)
+static __thread uint64_t *control_addr;
+static __thread uint64_t *control_addr_1;
+#else
+static uint64_t *control_addr;
+#endif
+
+static int page_size;
+
+
+static int init_channel_file(const char *base, const char *suffix, size_t size,
+ char **path, int *fd, uint64_t **addr, bool write)
+{
+int prot;
+
+*path = malloc(strlen(base) + strlen(suffix) + 1);
+sprintf(*path, "%s%s", base, suffix);
+
+prot = O_RDONLY;
+if (write) {
+prot = O_RDWR;
+}
+*fd = open(*path, prot);
+if (*fd == -1) {
+return -1;
+}
+
+prot = PROT_READ;
+if (write) {
+prot |= PROT_WRITE;
+}
+*addr = mmap(NULL, size, prot, MAP_SHARED, *fd, 0);
+if (*addr == MAP_FAILED) {
+return -1;
+}
+return 0;
+}
+
+#if !defined(CONFIG_USER_ONLY) && defined(__linux__)
+static int check_device_id(const char *base, const char *name, uint64_t value)
+{
+char tmp[1024];
+sprintf(tmp, "%s/%s", base, name);
+
+int fd = open(tmp, O_RDONLY);
+if (fd < 0) {
+return -1;
+}
+
+char v[1024];
+ssize_t s = read(fd, v, sizeof(v));
+if (s < 0) {
+close(fd);
+return -1;
+}
+v[s] = '\0';
+
+char *res;
+uint64_t vv = strtoull(v, &res, 16);
+if (*res == '\n' && vv == value) {
+return 0;
+} else {
+return -1;
+}
+}
+
+static char *f

[Qemu-devel] [PATCH v8 4/5] hypertrace: [softmmu] Add QEMU-side proxy to "guest_hypertrace" event

2017-07-30 Thread Lluís Vilanova
Uses a virtual device to trigger the hypertrace channel event.

Signed-off-by: Lluís Vilanova 
---
 hypertrace/Makefile.objs |4 +
 hypertrace/softmmu.c |  237 ++
 include/hw/pci/pci.h |2 
 3 files changed, 243 insertions(+)
 create mode 100644 hypertrace/softmmu.c

diff --git a/hypertrace/Makefile.objs b/hypertrace/Makefile.objs
index 177230fe1d..2ec1f04e79 100644
--- a/hypertrace/Makefile.objs
+++ b/hypertrace/Makefile.objs
@@ -1,10 +1,14 @@
 # -*- mode: makefile -*-
 
 target-obj-$(CONFIG_USER_ONLY) += user.o
+ifdef CONFIG_PCI
+target-obj-$(CONFIG_SOFTMMU) += softmmu.o
+endif
 target-obj-y += common.o
 
 $(obj)/user.o: $(obj)/emit.c
 $(obj)/common.o: $(obj)/emit.c
+$(obj)/softmmu.o: $(obj)/emit.c
 
 $(obj)/emit.c: $(obj)/emit.c-timestamp $(BUILD_DIR)/config-host.mak
@cmp $< $@ >/dev/null 2>&1 || cp $< $@
diff --git a/hypertrace/softmmu.c b/hypertrace/softmmu.c
new file mode 100644
index 00..4048c7e0ef
--- /dev/null
+++ b/hypertrace/softmmu.c
@@ -0,0 +1,237 @@
+/*
+ * QEMU-side management of hypertrace in softmmu emulation.
+ *
+ * Copyright (C) 2016-2017 Lluís Vilanova 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+/*
+ * Implementation details
+ * ==
+ *
+ * There are 3 channels, each a BAR of a virtual device that can be used 
through
+ * MMIO.
+ *
+ *
+ * - Configuration channel: Exposes configuration parameters.
+ *
+ * - Data channel: Lets guests write argument values. Each guest client should
+ *   use a different offset to avoid concurrency problems.
+ *
+ * - Control channel: Triggers the hypertrace event on a write, providing the
+ *   first argument. Offset in the control channel sets the offset in the data
+ *   channel.
+ */
+
+#include "qemu/osdep.h"
+#include "cpu.h"
+#include "hypertrace/common.h"
+#include "hypertrace/trace.h"
+#include "hw/pci/pci.h"
+#include "migration/blocker.h"
+#include "qapi/error.h"
+#include "qemu/error-report.h"
+
+
+typedef struct HypertraceState {
+PCIDevice dev;
+
+uint64_t max_clients;
+struct hypertrace_config hconfig;
+
+MemoryRegion config;
+void *config_ptr;
+MemoryRegion data;
+void *data_ptr;
+MemoryRegion control;
+void *control_ptr;
+
+Error *migration_blocker;
+} HypertraceState;
+
+
+static uint64_t hypertrace_control_io_read(void *opaque, hwaddr addr,
+   unsigned size)
+{
+uint64_t res;
+HypertraceState *s = opaque;
+char *mem = &((char *)s->control_ptr)[addr];
+
+switch (size) {
+case 1:
+{
+res = ((uint8_t *)mem)[0];
+break;
+}
+case 2:
+{
+res = ((uint16_t *)mem)[0];
+break;
+}
+case 4:
+{
+res = ((uint32_t *)mem)[0];
+break;
+}
+case 8:
+{
+res = ((uint64_t *)mem)[0];
+break;
+}
+default:
+error_report("error: hypertrace: Unexpected read of size %d", size);
+abort();
+}
+
+return res;
+}
+
+static void hypertrace_control_io_write(void *opaque, hwaddr addr,
+uint64_t data, unsigned size)
+{
+HypertraceState *s = opaque;
+char *mem = &((char *)s->control_ptr)[addr];
+
+switch (size) {
+case 1:
+{
+uint8_t *res = (uint8_t *)mem;
+*res = (uint8_t)data;
+break;
+}
+case 2:
+{
+uint16_t *res = (uint16_t *)mem;
+*res = (uint16_t)data;
+break;
+}
+case 4:
+{
+uint32_t *res = (uint32_t *)mem;
+*res = (uint32_t)data;
+break;
+}
+case 8:
+{
+uint64_t *res = (uint64_t *)mem;
+*res = (uint64_t)data;
+break;
+}
+default:
+error_report("error: hypertrace: Unexpected write of size %d", size);
+abort();
+}
+
+if ((addr + size) % sizeof(uint64_t) == 0) {
+uint64_t client = addr / sizeof(uint64_t);
+uint64_t vcontrol = ((uint64_t *)s->control_ptr)[client];
+uint64_t *data_ptr = (uint64_t *)s->data_ptr;
+data_ptr = &data_ptr[client * s->hconfig.client_data_size];
+hypertrace_emit(current_cpu, vcontrol, data_ptr);
+}
+}
+
+static const MemoryRegionOps hypertrace_control_ops = {
+.read = &hypertrace_control_io_read,
+.write = &hypertrace_control_io_write,
+.endianness = DEVICE_NATIVE_ENDIAN,
+.impl = {
+.min_access_size = 1,
+.max_access_size = 8,
+},
+};
+
+
+static void hypertrace_realize(PCIDevice *dev, Error **errp)
+{
+Error *local_err = NULL;
+struct hypertrace_config *config;
+HypertraceState *s = DO_UPCAST(HypertraceState, dev, dev);
+Error *err = NULL;
+
+if (s->max_clients < 1) {
+error_setg(errp, "hypertrace: must have at least one client");
+return;
+}
+
+hypertrace_init_config(&s->hconfig,

[Qemu-devel] [PATCH v8 3/5] hypertrace: [*-user] Add QEMU-side proxy to "guest_hypertrace" event

2017-07-30 Thread Lluís Vilanova
QEMU detects when the guest uses 'mmap' on hypertrace's control channel
file, and then uses 'mprotect' to detect accesses to it, which are used
to trigger tracing event "guest_hypertrace".

Signed-off-by: Lluís Vilanova 
---
 Makefile.objs|4 
 bsd-user/main.c  |   17 ++
 bsd-user/mmap.c  |   15 ++
 bsd-user/qemu.h  |3 
 bsd-user/syscall.c   |   34 ++--
 hypertrace/Makefile.objs |   21 ++
 hypertrace/common.c  |   55 ++
 hypertrace/common.h  |   25 +++
 hypertrace/user.c|  415 ++
 hypertrace/user.h|   71 
 include/qom/cpu.h|4 
 linux-user/main.c|   19 ++
 linux-user/mmap.c|   16 ++
 linux-user/qemu.h|3 
 linux-user/signal.c  |   12 +
 linux-user/syscall.c |   31 ++-
 16 files changed, 719 insertions(+), 26 deletions(-)
 create mode 100644 hypertrace/Makefile.objs
 create mode 100644 hypertrace/common.c
 create mode 100644 hypertrace/common.h
 create mode 100644 hypertrace/user.c
 create mode 100644 hypertrace/user.h

diff --git a/Makefile.objs b/Makefile.objs
index ce9a60137b..57479fa738 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -98,6 +98,10 @@ util-obj-y +=  trace/
 target-obj-y += trace/
 
 ##
+# hypertrace
+target-obj-y += hypertrace/
+
+##
 # guest agent
 
 # FIXME: a few definitions from qapi-types.o/qapi-visit.o are needed
diff --git a/bsd-user/main.c b/bsd-user/main.c
index fa9c012c9f..50df757209 100644
--- a/bsd-user/main.c
+++ b/bsd-user/main.c
@@ -30,9 +30,12 @@
 #include "tcg.h"
 #include "qemu/timer.h"
 #include "qemu/envlist.h"
+#include "qemu/error-report.h"
 #include "exec/log.h"
 #include "trace/control.h"
 #include "glib-compat.h"
+#include "hypertrace/user.h"
+
 
 int singlestep;
 unsigned long mmap_min_addr;
@@ -675,6 +678,8 @@ static void usage(void)
"-strace   log system calls\n"
"-trace
[[enable=]][,events=][,file=]\n"
"  specify tracing options\n"
+   "-hypertrace   [[base=]][,max-clients=]\n"
+   "  specify hypertrace options\n"
"\n"
"Environment variables:\n"
"QEMU_STRACE   Print system calls and arguments similar to 
the\n"
@@ -735,6 +740,8 @@ int main(int argc, char **argv)
 envlist_t *envlist = NULL;
 char *trace_file = NULL;
 bsd_type = target_openbsd;
+char *hypertrace_base = NULL;
+unsigned int hypertrace_max_clients = 0;
 
 if (argc <= 1)
 usage();
@@ -753,6 +760,7 @@ int main(int argc, char **argv)
 cpu_model = NULL;
 
 qemu_add_opts(&qemu_trace_opts);
+qemu_add_opts(&qemu_hypertrace_opts);
 
 optind = 1;
 for (;;) {
@@ -840,6 +848,10 @@ int main(int argc, char **argv)
 } else if (!strcmp(r, "trace")) {
 g_free(trace_file);
 trace_file = trace_opt_parse(optarg);
+} else if (!strcmp(r, "hypertrace")) {
+g_free(hypertrace_base);
+hypertrace_opt_parse(optarg, &hypertrace_base,
+ &hypertrace_max_clients);
 } else {
 usage();
 }
@@ -974,6 +986,11 @@ int main(int argc, char **argv)
 target_set_brk(info->brk);
 syscall_init();
 signal_init();
+if (atexit(hypertrace_fini) != 0) {
+error_report("error: atexit: %s", strerror(errno));
+abort();
+}
+hypertrace_init(hypertrace_base, hypertrace_max_clients);
 
 /* Now that we've loaded the binary, GUEST_BASE is fixed.  Delay
generating the prologue until now so that the prologue can take
diff --git a/bsd-user/mmap.c b/bsd-user/mmap.c
index 7f2018ede0..6a549a3553 100644
--- a/bsd-user/mmap.c
+++ b/bsd-user/mmap.c
@@ -21,6 +21,7 @@
 #include "qemu.h"
 #include "qemu-common.h"
 #include "bsd-mman.h"
+#include "hypertrace/user.h"
 
 //#define DEBUG_MMAP
 
@@ -240,10 +241,17 @@ static abi_ulong mmap_find_vma(abi_ulong start, abi_ulong 
size)
 return addr;
 }
 
-/* NOTE: all the constants are the HOST ones */
 abi_long target_mmap(abi_ulong start, abi_ulong len, int prot,
  int flags, int fd, abi_ulong offset)
 {
+return target_mmap_cpu(start, len, prot, flags, fd, offset, NULL);
+}
+
+/* NOTE: all the constants are the HOST ones */
+abi_long target_mmap_cpu(abi_ulong start, abi_ulong len, int prot,
+ int flags, int fd, abi_ulong offset,
+ CPUState *cpu)
+{
 abi_ulong ret, end, real_start, real_end, retaddr, host_offset, host_len;
 unsigned long host_start;
 
@@ -285,6 +293,10 @@ abi_long target_mmap(abi_ulong start, abi_ulong len, int 
prot,
 goto the_end;
 real_start = start & qemu_host_page_mask;
 
+if (!hypertrace_guest_mmap_check(fd, len, offset)) {
+

[Qemu-devel] [PATCH v8 2/5] hypertrace: Add tracing event "guest_hypertrace"

2017-07-30 Thread Lluís Vilanova
Generates the "guest_hypertrace" event with a user-configurable number
of arguments.

Signed-off-by: Lluís Vilanova 
---
 Makefile  |6 --
 Makefile.objs |2 ++
 configure |   34 ++
 rules.mak |2 +-
 trace-events  |   11 +++
 5 files changed, 52 insertions(+), 3 deletions(-)

diff --git a/Makefile b/Makefile
index 38814f9a61..f08f498f79 100644
--- a/Makefile
+++ b/Makefile
@@ -84,6 +84,7 @@ endif
 GENERATED_FILES += $(TRACE_HEADERS)
 GENERATED_FILES += $(TRACE_SOURCES)
 GENERATED_FILES += $(BUILD_DIR)/trace-events-all
+GENERATED_FILES += $(BUILD_DIR)/hypertrace/trace.h
 
 trace-group-name = $(shell dirname $1 | sed -e 's/[^a-zA-Z0-9]/_/g')
 
@@ -92,7 +93,7 @@ tracetool-y += $(shell find $(SRC_PATH)/scripts/tracetool 
-name "*.py")
 
 %/trace.h: %/trace.h-timestamp
@cmp $< $@ >/dev/null 2>&1 || cp $< $@
-%/trace.h-timestamp: $(SRC_PATH)/%/trace-events $(tracetool-y)
+%/trace.h-timestamp: %/trace-events $(tracetool-y)
$(call quiet-command,$(TRACETOOL) \
--group=$(call trace-group-name,$@) \
--format=h \
@@ -101,7 +102,7 @@ tracetool-y += $(shell find $(SRC_PATH)/scripts/tracetool 
-name "*.py")
 
 %/trace.c: %/trace.c-timestamp
@cmp $< $@ >/dev/null 2>&1 || cp $< $@
-%/trace.c-timestamp: $(SRC_PATH)/%/trace-events $(tracetool-y)
+%/trace.c-timestamp: %/trace-events $(tracetool-y)
$(call quiet-command,$(TRACETOOL) \
--group=$(call trace-group-name,$@) \
--format=c \
@@ -534,6 +535,7 @@ distclean: clean
rm -Rf .sdk
if test -f pixman/config.log; then $(MAKE) -C pixman distclean; fi
if test -f dtc/version_gen.h; then $(MAKE) $(DTC_MAKE_ARGS) clean; fi
+   rm -f hypertrace/trace-events
 
 KEYMAPS=da en-gb  et  fr fr-ch  is  lt  modifiers  no  pt-br  sv \
 ar  de en-us  fi  fr-be  hr it  lv  nl pl  ru th \
diff --git a/Makefile.objs b/Makefile.objs
index bfd5a6ceb1..ce9a60137b 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -170,9 +170,11 @@ trace-events-subdirs += accel/kvm
 trace-events-subdirs += nbd
 
 trace-events-files = $(SRC_PATH)/trace-events 
$(trace-events-subdirs:%=$(SRC_PATH)/%/trace-events)
+trace-events-files += $(BUILD_DIR)/hypertrace/trace-events
 
 trace-obj-y = trace-root.o
 trace-obj-y += $(trace-events-subdirs:%=%/trace.o)
 trace-obj-$(CONFIG_TRACE_UST) += trace-ust-all.o
 trace-obj-$(CONFIG_TRACE_DTRACE) += trace-dtrace-root.o
 trace-obj-$(CONFIG_TRACE_DTRACE) += $(trace-events-subdirs:%=%/trace-dtrace.o)
+trace-obj-y += $(BUILD_DIR)/hypertrace/trace.o
diff --git a/configure b/configure
index a3f0522e8f..e6f752c242 100755
--- a/configure
+++ b/configure
@@ -351,6 +351,7 @@ pie=""
 qom_cast_debug="yes"
 trace_backends="log"
 trace_file="trace"
+hypertrace="1"
 spice=""
 rbd=""
 smartcard=""
@@ -895,6 +896,8 @@ for opt do
   ;;
   --with-trace-file=*) trace_file="$optarg"
   ;;
+  --with-hypertrace-args=*) hypertrace="$optarg"
+  ;;
   --enable-gprof) gprof="yes"
   ;;
   --enable-gcov) gcov="yes"
@@ -1425,6 +1428,8 @@ Advanced options (experts only):
Available backends: $trace_backend_list
   --with-trace-file=NAME   Full PATH,NAME of file to store traces
Default:trace-
+  --with-hypertrace-args=NUMBER
+   number of hypertrace arguments (default: 
$hypertrace)
   --disable-slirp  disable SLIRP userspace network connectivity
   --enable-tcg-interpreter enable TCG with bytecode interpreter (TCI)
   --oss-libpath to OSS library
@@ -4434,6 +4439,16 @@ if test "$?" -ne 0 ; then
 fi
 
 ##
+# check hypertrace arguments
+case "$hypertrace" in
+''|*[!0-9]*) error_exit "invalid number of hypertrace arguments" ;;
+*) ;;
+esac
+if test $hypertrace = 0; then
+error_exit "hypertrace arguments must be greater than zero"
+fi
+
+##
 # For 'ust' backend, test if ust headers are present
 if have_backend "ust"; then
   cat > $TMPC << EOF
@@ -5282,6 +5297,7 @@ echo "Trace backends$trace_backends"
 if have_backend "simple"; then
 echo "Trace output file $trace_file-"
 fi
+echo "Hypertrace arguments  $hypertrace"
 echo "spice support $spice $(echo_version $spice 
$spice_protocol_version/$spice_server_version)"
 echo "rbd support   $rbd"
 echo "xfsctl support$xfs"
@@ -5989,6 +6005,24 @@ else
 fi
 QEMU_INCLUDES="-I\$(SRC_PATH)/tcg $QEMU_INCLUDES"
 
+# hypertrace
+echo "CONFIG_HYPERTRACE_ARGS=$hypertrace" >> $config_host_mak
+hypertrace_events=hypertrace/trace-events
+mkdir -p $(dirname $hypertrace_events)
+echo "# See docs/trace-events.txt for syntax documentation." 
>$hypertrace_events
+echo -n 'vcpu guest_hypertrace(' >>$hypertrace_events
+for i in `seq $hypertrace`; do
+if test $i != 1; then
+echo -n ", " >>$hypertrace_events
+fi
+echo -n "uint64_t arg$i" >>$hypertrac

[Qemu-devel] [PATCH v8 1/5] hypertrace: Add documentation

2017-07-30 Thread Lluís Vilanova
Signed-off-by: Lluís Vilanova 
---
 docs/devel/tracing.txt |3 +
 docs/hypertrace.txt|  225 
 2 files changed, 228 insertions(+)
 create mode 100644 docs/hypertrace.txt

diff --git a/docs/devel/tracing.txt b/docs/devel/tracing.txt
index 5768a0b7a2..9178a308da 100644
--- a/docs/devel/tracing.txt
+++ b/docs/devel/tracing.txt
@@ -5,6 +5,9 @@
 This document describes the tracing infrastructure in QEMU and how to use it
 for debugging, profiling, and observing execution.
 
+See "docs/hypertrace.txt" to correlate guest tracing events with those in the
+QEMU host.
+
 == Quickstart ==
 
 1. Build with the 'simple' trace backend:
diff --git a/docs/hypertrace.txt b/docs/hypertrace.txt
new file mode 100644
index 00..c3715db25b
--- /dev/null
+++ b/docs/hypertrace.txt
@@ -0,0 +1,225 @@
+= Hypertrace channel =
+
+Copyright (C) 2016-2017 Lluís Vilanova 
+
+This work is licensed under the terms of the GNU GPL, version 2 or later.
+See the COPYING file in the top-level directory.
+
+
+The hypertrace channel allows guest code to emit events in QEMU (the host) 
using
+its tracing infrastructure (see "docs/trace.txt"). This works in both 'system'
+and 'user' modes. Therefore, hypertrace is to tracing what hypercalls are to
+system calls.
+
+The hypertrace channel can be used for various purposes:
+
+* Using guest code semantics to guide which QEMU events to trace at each point
+  in time. The example "Quick guide" below shows how to use this to identify
+  "regions of interest" in your guest code. It then uses these regions to trace
+  QEMU's behaviour during their execution, without paying the price of tracing
+  events outside the interest regions.
+
+* Mark "progress points" in guest code (e.g., processed client requests,
+  scheduled processes, etc), so that they can be easily traced and correlated
+  between QEMU's various tracing events and the guest's own tracing
+  infrastructure (e.g., Linux's tracepoints).
+
+* You can also use regions of interest and progress points on the guest code to
+  time the performance of new TCG optimizations. Each hypertrace event comes
+  with a host timestamp, making it easy to compare the host execution times of
+  interesting guest code.
+
+Hypertrace features:
+
+* Works with 'system' and 'user' mode.
+
+* Minimal setup for the guest; QEMU provides support guest code libraries that
+  work out of the box.
+
+* Independent of guest architecture; the guest code uses accesses to special
+  memory regions, as opposed to redefining instruction semantics.
+
+* Negligible guest overhead; emitting a hypertrace event requires a single 
guest
+  memory access, making it as unobtrusive as possible.
+
+Warning: The hypertrace channel in 'system' mode only works in systems with
+support for PCI. You can get the list of guests with PCI support with 'grep
+pci.mak default-configs/*'.
+
+
+== Quick guide ==
+
+This shows an example of using the hypertrace channel to trace the guest memory
+accesses only in a specific guest code region, which is identified by calls to
+the hypertrace channel.
+
+We are going to trace memory accesses to disk using QEMU's "log" backend, and
+will use QEMU's "dtrace" backend (SystemTap) to ensure memory accesses are only
+traced in the guest code region of interest. The first time the guest code
+invokes the hypertrace channel, we will start tracing the
+"guest_mem_before_exec" event using dtrace, and then will disable it the second
+time around.
+
+Tracing is done with "log" because it is more efficient than using "dtrace" in
+high-volume events like memory accesses.
+
+1. Set the tracing backends and number of arguments for the hypertrace events:
+
+mkdir /tmp/qemu-build
+cd /tmp/qemu-build
+/path/to/qemu-source/configure  \
+--enable-trace-backends=dtrace,log  \
+--with-hypertrace-args=4\
+--prefix=/tmp/qemu-install
+make -j install
+
+2. Compile QEMU:
+
+make -C /tmp/qemu-build install -j
+
+3. Compile the guest support code:
+
+make -C /tmp/qemu-build/x86_64-linux-user/hypertrace/guest
+make -C /tmp/qemu-build/x86_64-softmmu/hypertrace/guest
+
+   If you need to cross-compile the guest library, set the 'CC' variable:
+
+make -C /tmp/qemu-build/mipsel-linux-user/hypertrace/guest 
CC=mipsel-gnu-linux-gcc
+
+4. Create a guest application that interacts with the hypertrace channel:
+
+cat > /tmp/my-hypertrace.c <<\EOF
+#include 
+#include 
+#include 
+#include 
+#include 
+
+
+int main(int argc, char **argv)
+{
+char *base = NULL;
+if (argc > 1) {
+base = argv[1];
+}
+
+/* In 'user' mode this path must be the same we will use to start 
QEMU. */
+if (qemu_hypertrace_init(base) != 0) {
+perror("error: qemu_hypertrace_init");
+abort();
+}
+
+/* Set additional event arguments (unused in 

[Qemu-devel] [PATCH v8 0/5] hypertrace: Lightweight guest-to-QEMU trace channel

2017-07-30 Thread Lluís Vilanova
The hypertrace channel allows guest code to emit events in QEMU (the host) using
its tracing infrastructure (see "docs/trace.txt"). This works in both 'system'
and 'user' modes, is architecture-agnostic and introduces minimal noise on the
guest.

See first commit for a full description, use-cases and an example.

Signed-off-by: Lluís Vilanova 
---

Changes in v8
=

* Do not use 'seq' when there's no extra hypertrace arguments (BSD behaves
  differently for "seq 0").
* Fix compilation for bsd-user.


Changes in v7
=

* Use 'expr' instead of assuming 'bash' when generating the "emit.c" file.
* Restore generation of trace-events-all.


Changes in v6
=

* Fix compilation errors.


Changes in v5
=

* Rebase on 5a477a7806.
* Fix typo in "bsd-user/main.c" [Stephan Hajnoczi].
* Replace abort() with exit() in command-line errors [Stephan Hajnoczi].
* Fix alignment of data and control channels [Stephan Hajnoczi].
* Fix signal reflection in user-mode (SIGINT, SIGABRT, SIGSEGV) [Stephan 
Hajnoczi].
* Clarify semantics of hypertrace_guest_mmap_check() [Stephan Hajnoczi].
* Use uintptr_t instead of unsigned long in SEGV handler [Stephan Hajnoczi].
* Emit hypertrace's event with host-endian arguments [Stephan Hajnoczi].
* Enable true concurrency between user-mode guest threads by using a spearate 
control channel page per client [Stephan Hajnoczi].
* Remove unused PAGE_SIZE define [Stephan Hajnoczi].
* Submit linux kernel API module separately to Linux upstream [Stephan 
Hajnoczi].
* Assume guest code events are always enabled.


Changes in v4
=

* Fix typo in stap script example.
* Fix compilation instructions in doc/hypertrace.txt.
* Rebase on 0737f32daf.


Changes in v3
=

* Rebase on 4a58f35.
* Remove debugging printf's.
* Fix style issues identified by checkpatch.
* Fix control channel mapping in guest linux module.
* Add a short event description in "trace-events".
* Polish documentation in 1st patch.


Changes in v2
=

* Remove unnecessary casts for g2h() [Eric Blake].
* Use perror() [Eric Blake].
* Avoid expansions in application example [Eric Blake].
* Add copyright in document "hypertrace.txt" [Eric Blake].
* Make the user-mode hypertrace invocations thread-safe [Stefan Hajnoczi].
* Split dynamic hypertrace configuration into a separate "config" channel.

Lluís Vilanova (5):
  hypertrace: Add documentation
  hypertrace: Add tracing event "guest_hypertrace"
  hypertrace: [*-user] Add QEMU-side proxy to "guest_hypertrace" event
  hypertrace: [softmmu] Add QEMU-side proxy to "guest_hypertrace" event
  hypertrace: Add guest-side user-level library


 Makefile   |   11 +
 Makefile.objs  |6 +
 bsd-user/main.c|   17 +
 bsd-user/mmap.c|   15 +
 bsd-user/qemu.h|3 
 bsd-user/syscall.c |   34 ++-
 configure  |   36 +++
 docs/devel/tracing.txt |3 
 docs/hypertrace.txt|  225 
 hypertrace/Makefile.objs   |   25 ++
 hypertrace/common.c|   55 +
 hypertrace/common.h|   25 ++
 hypertrace/guest/Makefile  |   30 +++
 hypertrace/guest/common.c  |  301 ++
 hypertrace/guest/qemu-hypertrace.h |   80 +++
 hypertrace/softmmu.c   |  237 +
 hypertrace/user.c  |  415 
 hypertrace/user.h  |   71 ++
 include/hw/pci/pci.h   |2 
 include/qom/cpu.h  |4 
 linux-user/main.c  |   19 ++
 linux-user/mmap.c  |   16 +
 linux-user/qemu.h  |3 
 linux-user/signal.c|   12 +
 linux-user/syscall.c   |   31 ++-
 rules.mak  |2 
 trace-events   |   11 +
 27 files changed, 1660 insertions(+), 29 deletions(-)
 create mode 100644 docs/hypertrace.txt
 create mode 100644 hypertrace/Makefile.objs
 create mode 100644 hypertrace/common.c
 create mode 100644 hypertrace/common.h
 create mode 100644 hypertrace/guest/Makefile
 create mode 100644 hypertrace/guest/common.c
 create mode 100644 hypertrace/guest/qemu-hypertrace.h
 create mode 100644 hypertrace/softmmu.c
 create mode 100644 hypertrace/user.c
 create mode 100644 hypertrace/user.h


To: qemu-devel@nongnu.org
Cc: Stefan Hajnoczi 
Cc: Eric Blake 
Cc: Luiz Capitulino 
Cc: Daniel P Berrange 



Re: [Qemu-devel] [PATCH v7 0/5] hypertrace: Lightweight guest-to-QEMU trace channel

2017-07-30 Thread Lluís Vilanova
Lluís Vilanova writes:

> no-reply  writes:
>> Hi,
>> This series failed build test on FreeBSD host. Please find the details below.

>> Subject: [Qemu-devel] [PATCH v7 0/5] hypertrace: Lightweight guest-to-QEMU 
>> trace channel
>> Type: series
>> Message-id: 150141363372.9857.9055094189388759132.st...@frigg.lan

>> === TEST SCRIPT BEGIN ===
>> #!/bin/sh
>> # Testing script will be invoked under the git checkout with
>> # HEAD pointing to a commit that has the patches applied on top of "base"
>> # branch
>> set -e
>> echo "=== ENV ==="
>> env
>> echo "=== PACKAGES ==="
>> pkg info
>> echo "=== TEST BEGIN ==="
>> CC=/usr/local/libexec/ccache/cc
>> INSTALL=$PWD/install
>> BUILD=$PWD/build
>> echo -n "Using CC: "
>> realpath $CC
>> mkdir -p $BUILD $INSTALL
>> SRC=$PWD
>> cd $BUILD
>> $SRC/configure --cc=$CC --prefix=$INSTALL --target-list=x86_64-softmmu
>> gmake -j4
>> # XXX: we need reliable clean up
>> # make check -j4 V=1
>> gmake install
>> === TEST SCRIPT END ===

>> Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
>> From https://github.com/patchew-project/qemu
>> * [new tag]   
>> patchew/150141363372.9857.9055094189388759132.st...@frigg.lan -> 
>> patchew/150141363372.9857.9055094189388759132.st...@frigg.lan
>> Switched to a new branch 'test'
>> 5dad7c93de hypertrace: Add guest-side user-level library
>> dcf9d9c7c7 hypertrace: [softmmu] Add QEMU-side proxy to "guest_hypertrace" 
>> event
>> 4c041f724d hypertrace: [*-user] Add QEMU-side proxy to "guest_hypertrace" 
>> event
>> 73796a3885 hypertrace: Add tracing event "guest_hypertrace"
>> 43fe7e2e2e hypertrace: Add documentation

>> === OUTPUT BEGIN ===
>> === ENV ===
>> LOGNAME=patchew-tester
>> PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin
>> PWD=/var/tmp/patchew-tester-tmp-eswc2rbf/src
>> HOME=/home/patchew-tester
>> USER=patchew-tester
>> SHELL=/bin/sh
>> PATCHEW=/home/patchew-tester/patchew/patchew-cli -s http://patchew.org 
>> --nodebug
>> === PACKAGES ===
>> bash-4.4.12_2  GNU Project's Bourne Again SHell
>> binutils-2.28,1GNU binary tools
>> bison-3.0.4,1  Parser generator from FSF, (mostly) 
>> compatible with Yacc
>> ca_root_nss-3.30.2 Root certificate bundle from the Mozilla 
>> Project
>> ccache-3.3.4_3 Tool to minimize the compile time of C/C++ 
>> programs
>> curl-7.54.0Non-interactive tool to get files from FTP, 
>> GOPHER, HTTP(S) servers
>> cvsps-2.1_2Create patchset information from CVS
>> dtc-1.4.2_1Device Tree Compiler
>> expat-2.2.0_1  XML 1.0 parser written in C
>> gcc-5.4.0  GNU Compiler Collection 5
>> gcc-ecj-4.5Eclipse Java Compiler used to build GCC Java
>> gettext-runtime-0.19.8.1_1 GNU gettext runtime libraries and programs
>> git-2.13.0 Distributed source code management tool
>> glib-2.46.2_5  Some useful routines of C programming 
>> (current stable version)
>> gmake-4.2.1_1  GNU version of 'make' utility
>> gmp-6.1.2  Free library for arbitrary precision 
>> arithmetic
>> indexinfo-0.2.6Utility to regenerate the GNU info page index
>> libffi-3.2.1   Foreign Function Interface
>> libiconv-1.14_10   Character set conversion library
>> libnghttp2-1.21.0  HTTP/2.0 C Library
>> m4-1.4.18,1GNU M4
>> mpc-1.0.3  Library of complex numbers with arbitrarily 
>> high precision
>> mpfr-3.1.5_1   Library for multiple-precision floating-point 
>> computations
>> p5-Authen-SASL-2.16_1  Perl5 module for SASL authentication
>> p5-Digest-HMAC-1.03_1  Perl5 interface to HMAC Message-Digest 
>> Algorithms
>> p5-Error-0.17024   Error/exception handling in object-oriented 
>> programming style
>> p5-GSSAPI-0.28_1   Perl extension providing access to the 
>> GSSAPIv2 library
>> pcre-8.40  Perl Compatible Regular Expressions library
>> perl5-5.24.1   Practical Extraction and Report Language
>> pixman-0.34.0  Low-level pixel manipulation library
>> pkg-1.10.1 Package manager
>> pkgconf-1.3.0,1Utility to help to configure compiler and 
>> linker flags
>> python-2.7_3,2 "meta-port" for the default version of Python 
>> interpreter
>> python2-2_3The "meta-port" for version 2 of the Python 
>> interpreter
>> python27-2.7.13_3  Interpreted object-oriented programming 
>> language
>> python3-3_3The "meta-port" for version 3 of the Python 
>> interpreter
>> python35-3.5.3_1   Interpreted object-oriented programming 
>> language
>> readline-6.3.8 Library for editing command lines as they are 
>> typed
>> sudo-1.8.20p1   

Re: [Qemu-devel] [PATCH v7 0/5] hypertrace: Lightweight guest-to-QEMU trace channel

2017-07-30 Thread Lluís Vilanova
no-reply  writes:

> Hi,
> This series failed build test on FreeBSD host. Please find the details below.

> Subject: [Qemu-devel] [PATCH v7 0/5] hypertrace: Lightweight guest-to-QEMU 
> trace channel
> Type: series
> Message-id: 150141363372.9857.9055094189388759132.st...@frigg.lan

> === TEST SCRIPT BEGIN ===
> #!/bin/sh
> # Testing script will be invoked under the git checkout with
> # HEAD pointing to a commit that has the patches applied on top of "base"
> # branch
> set -e
> echo "=== ENV ==="
> env
> echo "=== PACKAGES ==="
> pkg info
> echo "=== TEST BEGIN ==="
> CC=/usr/local/libexec/ccache/cc
> INSTALL=$PWD/install
> BUILD=$PWD/build
> echo -n "Using CC: "
> realpath $CC
> mkdir -p $BUILD $INSTALL
> SRC=$PWD
> cd $BUILD
> $SRC/configure --cc=$CC --prefix=$INSTALL --target-list=x86_64-softmmu
> gmake -j4
> # XXX: we need reliable clean up
> # make check -j4 V=1
> gmake install
> === TEST SCRIPT END ===

> Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
> From https://github.com/patchew-project/qemu
>  * [new tag]   
> patchew/150141363372.9857.9055094189388759132.st...@frigg.lan -> 
> patchew/150141363372.9857.9055094189388759132.st...@frigg.lan
> Switched to a new branch 'test'
> 5dad7c93de hypertrace: Add guest-side user-level library
> dcf9d9c7c7 hypertrace: [softmmu] Add QEMU-side proxy to "guest_hypertrace" 
> event
> 4c041f724d hypertrace: [*-user] Add QEMU-side proxy to "guest_hypertrace" 
> event
> 73796a3885 hypertrace: Add tracing event "guest_hypertrace"
> 43fe7e2e2e hypertrace: Add documentation

> === OUTPUT BEGIN ===
> === ENV ===
> LOGNAME=patchew-tester
> PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin
> PWD=/var/tmp/patchew-tester-tmp-eswc2rbf/src
> HOME=/home/patchew-tester
> USER=patchew-tester
> SHELL=/bin/sh
> PATCHEW=/home/patchew-tester/patchew/patchew-cli -s http://patchew.org 
> --nodebug
> === PACKAGES ===
> bash-4.4.12_2  GNU Project's Bourne Again SHell
> binutils-2.28,1GNU binary tools
> bison-3.0.4,1  Parser generator from FSF, (mostly) compatible 
> with Yacc
> ca_root_nss-3.30.2 Root certificate bundle from the Mozilla 
> Project
> ccache-3.3.4_3 Tool to minimize the compile time of C/C++ 
> programs
> curl-7.54.0Non-interactive tool to get files from FTP, 
> GOPHER, HTTP(S) servers
> cvsps-2.1_2Create patchset information from CVS
> dtc-1.4.2_1Device Tree Compiler
> expat-2.2.0_1  XML 1.0 parser written in C
> gcc-5.4.0  GNU Compiler Collection 5
> gcc-ecj-4.5Eclipse Java Compiler used to build GCC Java
> gettext-runtime-0.19.8.1_1 GNU gettext runtime libraries and programs
> git-2.13.0 Distributed source code management tool
> glib-2.46.2_5  Some useful routines of C programming (current 
> stable version)
> gmake-4.2.1_1  GNU version of 'make' utility
> gmp-6.1.2  Free library for arbitrary precision arithmetic
> indexinfo-0.2.6Utility to regenerate the GNU info page index
> libffi-3.2.1   Foreign Function Interface
> libiconv-1.14_10   Character set conversion library
> libnghttp2-1.21.0  HTTP/2.0 C Library
> m4-1.4.18,1GNU M4
> mpc-1.0.3  Library of complex numbers with arbitrarily 
> high precision
> mpfr-3.1.5_1   Library for multiple-precision floating-point 
> computations
> p5-Authen-SASL-2.16_1  Perl5 module for SASL authentication
> p5-Digest-HMAC-1.03_1  Perl5 interface to HMAC Message-Digest 
> Algorithms
> p5-Error-0.17024   Error/exception handling in object-oriented 
> programming style
> p5-GSSAPI-0.28_1   Perl extension providing access to the 
> GSSAPIv2 library
> pcre-8.40  Perl Compatible Regular Expressions library
> perl5-5.24.1   Practical Extraction and Report Language
> pixman-0.34.0  Low-level pixel manipulation library
> pkg-1.10.1 Package manager
> pkgconf-1.3.0,1Utility to help to configure compiler and 
> linker flags
> python-2.7_3,2 "meta-port" for the default version of Python 
> interpreter
> python2-2_3The "meta-port" for version 2 of the Python 
> interpreter
> python27-2.7.13_3  Interpreted object-oriented programming 
> language
> python3-3_3The "meta-port" for version 3 of the Python 
> interpreter
> python35-3.5.3_1   Interpreted object-oriented programming 
> language
> readline-6.3.8 Library for editing command lines as they are 
> typed
> sudo-1.8.20p1  Allow others to run commands as root
> === TEST BEGIN ===
> Using CC: /usr/local/bin/ccache
> Install prefix/var/tmp/patchew-tes

Re: [Qemu-devel] [PATCH v7 0/5] hypertrace: Lightweight guest-to-QEMU trace channel

2017-07-30 Thread Lluís Vilanova
no-reply  writes:

> Hi,
> This series seems to have some coding style problems. See output below for
> more information:

> Subject: [Qemu-devel] [PATCH v7 0/5] hypertrace: Lightweight guest-to-QEMU 
> trace channel
> Message-id: 150141363372.9857.9055094189388759132.st...@frigg.lan
> Type: series

> === TEST SCRIPT BEGIN ===
> #!/bin/bash

> BASE=base
> n=1
> total=$(git log --oneline $BASE.. | wc -l)
> failed=0

> git config --local diff.renamelimit 0
> git config --local diff.renames True

> commits="$(git log --format=%H --reverse $BASE..)"
> for c in $commits; do
> echo "Checking PATCH $n/$total: $(git log -n 1 --format=%s $c)..."
> if ! git show $c --format=email | ./scripts/checkpatch.pl --mailback -; 
> then
> failed=1
> echo
> fi
> n=$((n+1))
> done

> exit $failed
> === TEST SCRIPT END ===

> Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
> From https://github.com/patchew-project/qemu
>  * [new tag]   
> patchew/1501444898-27668-1-git-send-email-peng.h...@zte.com.cn -> 
> patchew/1501444898-27668-1-git-send-email-peng.h...@zte.com.cn
> Switched to a new branch 'test'
> 5dad7c93de hypertrace: Add guest-side user-level library
> dcf9d9c7c7 hypertrace: [softmmu] Add QEMU-side proxy to "guest_hypertrace" 
> event
> 4c041f724d hypertrace: [*-user] Add QEMU-side proxy to "guest_hypertrace" 
> event
> 73796a3885 hypertrace: Add tracing event "guest_hypertrace"
> 43fe7e2e2e hypertrace: Add documentation

> === OUTPUT BEGIN ===
> Checking PATCH 1/5: hypertrace: Add documentation...
> Checking PATCH 2/5: hypertrace: Add tracing event "guest_hypertrace"...
> Checking PATCH 3/5: hypertrace: [*-user] Add QEMU-side proxy to 
> "guest_hypertrace" event...
> Checking PATCH 4/5: hypertrace: [softmmu] Add QEMU-side proxy to 
> "guest_hypertrace" event...
> Checking PATCH 5/5: hypertrace: Add guest-side user-level library...
> WARNING: architecture specific defines should be avoided
> #171: FILE: hypertrace/guest/common.c:84:
> +#if !defined(CONFIG_USER_ONLY) && defined(__linux__)

> ERROR: consider using qemu_strtoull in preference to strtoull
> #191: FILE: hypertrace/guest/common.c:104:
> +uint64_t vv = strtoull(v, &res, 16);

> total: 1 errors, 1 warnings, 432 lines checked

> Your patch has style problems, please review.  If any of these errors
> are false positives report them to the maintainer, see
> CHECKPATCH in MAINTAINERS.

> === OUTPUT END ===

> Test command exited with code: 1

I think both are false positives for this specific case.

Cheers,
  Lluis



Re: [Qemu-devel] [PATCH] target-i386 : fix a race condition result to lost INIT

2017-07-30 Thread Wanpeng Li
2017-07-31 4:01 GMT+08:00 Peng Hao :
> when SMP VM start, AP may lost INIT because of receiving INIT between
> kvm_vcpu_ioctl_x86_get/set_vcpu_events.
>
>vcpu 0 vcpu 1
>kvm_vcpu_ioctl_x86_get_vcpu_events
>events->smi.latched_init=0
>  send INIT to vcpu1
>set vcpu1's pending_events
>kvm_vcpu_ioctl_x86_set_vcpu_events
> events->smi.latched_init == 0
>   clear INIT in pending_events
> considering migration, just at level >= KVM_PUT_RESET_STATE,
> add KVM_VCPUEVENT_VALID_SMM in events.flags.I think it is better to
> modify in qemu.

Do it in qemu just avoids to trigger the bug instead of fixing it. Why
we touch INIT pending if INIT is not latched in SMM?

Regards,
Wanpeng Li

>
> Signed-off-by: Peng Hao 
> ---
>  target/i386/kvm.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
> index a6613e1..be146cb 100644
> --- a/target/i386/kvm.c
> +++ b/target/i386/kvm.c
> @@ -2438,7 +2438,7 @@ static int kvm_put_vcpu_events(X86CPU *cpu, int level)
>  /* Stop SMI delivery on old machine types to avoid a reboot
>   * on an inward migration of an old VM.
>   */
> -if (!cpu->kvm_no_smi_migration) {
> +if (!cpu->kvm_no_smi_migration && (level >= KVM_PUT_RESET_STATE)) {
>  events.flags |= KVM_VCPUEVENT_VALID_SMM;
>  }
>  }
> --
> 1.8.3.1
>
>



Re: [Qemu-devel] [PATCH v7 0/5] hypertrace: Lightweight guest-to-QEMU trace channel

2017-07-30 Thread no-reply
Hi,

This series seems to have some coding style problems. See output below for
more information:

Subject: [Qemu-devel] [PATCH v7 0/5] hypertrace: Lightweight guest-to-QEMU 
trace channel
Message-id: 150141363372.9857.9055094189388759132.st...@frigg.lan
Type: series

=== TEST SCRIPT BEGIN ===
#!/bin/bash

BASE=base
n=1
total=$(git log --oneline $BASE.. | wc -l)
failed=0

git config --local diff.renamelimit 0
git config --local diff.renames True

commits="$(git log --format=%H --reverse $BASE..)"
for c in $commits; do
echo "Checking PATCH $n/$total: $(git log -n 1 --format=%s $c)..."
if ! git show $c --format=email | ./scripts/checkpatch.pl --mailback -; then
failed=1
echo
fi
n=$((n+1))
done

exit $failed
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 * [new tag]   
patchew/1501444898-27668-1-git-send-email-peng.h...@zte.com.cn -> 
patchew/1501444898-27668-1-git-send-email-peng.h...@zte.com.cn
Switched to a new branch 'test'
5dad7c93de hypertrace: Add guest-side user-level library
dcf9d9c7c7 hypertrace: [softmmu] Add QEMU-side proxy to "guest_hypertrace" event
4c041f724d hypertrace: [*-user] Add QEMU-side proxy to "guest_hypertrace" event
73796a3885 hypertrace: Add tracing event "guest_hypertrace"
43fe7e2e2e hypertrace: Add documentation

=== OUTPUT BEGIN ===
Checking PATCH 1/5: hypertrace: Add documentation...
Checking PATCH 2/5: hypertrace: Add tracing event "guest_hypertrace"...
Checking PATCH 3/5: hypertrace: [*-user] Add QEMU-side proxy to 
"guest_hypertrace" event...
Checking PATCH 4/5: hypertrace: [softmmu] Add QEMU-side proxy to 
"guest_hypertrace" event...
Checking PATCH 5/5: hypertrace: Add guest-side user-level library...
WARNING: architecture specific defines should be avoided
#171: FILE: hypertrace/guest/common.c:84:
+#if !defined(CONFIG_USER_ONLY) && defined(__linux__)

ERROR: consider using qemu_strtoull in preference to strtoull
#191: FILE: hypertrace/guest/common.c:104:
+uint64_t vv = strtoull(v, &res, 16);

total: 1 errors, 1 warnings, 432 lines checked

Your patch has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

=== OUTPUT END ===

Test command exited with code: 1


---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-de...@freelists.org

Re: [Qemu-devel] [PATCH v7 0/5] hypertrace: Lightweight guest-to-QEMU trace channel

2017-07-30 Thread no-reply
Hi,

This series failed build test on FreeBSD host. Please find the details below.

Subject: [Qemu-devel] [PATCH v7 0/5] hypertrace: Lightweight guest-to-QEMU 
trace channel
Type: series
Message-id: 150141363372.9857.9055094189388759132.st...@frigg.lan

=== TEST SCRIPT BEGIN ===
#!/bin/sh
# Testing script will be invoked under the git checkout with
# HEAD pointing to a commit that has the patches applied on top of "base"
# branch
set -e
echo "=== ENV ==="
env
echo "=== PACKAGES ==="
pkg info
echo "=== TEST BEGIN ==="
CC=/usr/local/libexec/ccache/cc
INSTALL=$PWD/install
BUILD=$PWD/build
echo -n "Using CC: "
realpath $CC
mkdir -p $BUILD $INSTALL
SRC=$PWD
cd $BUILD
$SRC/configure --cc=$CC --prefix=$INSTALL --target-list=x86_64-softmmu
gmake -j4
# XXX: we need reliable clean up
# make check -j4 V=1
gmake install
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 * [new tag]   
patchew/150141363372.9857.9055094189388759132.st...@frigg.lan -> 
patchew/150141363372.9857.9055094189388759132.st...@frigg.lan
Switched to a new branch 'test'
5dad7c93de hypertrace: Add guest-side user-level library
dcf9d9c7c7 hypertrace: [softmmu] Add QEMU-side proxy to "guest_hypertrace" event
4c041f724d hypertrace: [*-user] Add QEMU-side proxy to "guest_hypertrace" event
73796a3885 hypertrace: Add tracing event "guest_hypertrace"
43fe7e2e2e hypertrace: Add documentation

=== OUTPUT BEGIN ===
=== ENV ===
LOGNAME=patchew-tester
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin
PWD=/var/tmp/patchew-tester-tmp-eswc2rbf/src
HOME=/home/patchew-tester
USER=patchew-tester
SHELL=/bin/sh
PATCHEW=/home/patchew-tester/patchew/patchew-cli -s http://patchew.org --nodebug
=== PACKAGES ===
bash-4.4.12_2  GNU Project's Bourne Again SHell
binutils-2.28,1GNU binary tools
bison-3.0.4,1  Parser generator from FSF, (mostly) compatible 
with Yacc
ca_root_nss-3.30.2 Root certificate bundle from the Mozilla Project
ccache-3.3.4_3 Tool to minimize the compile time of C/C++ 
programs
curl-7.54.0Non-interactive tool to get files from FTP, 
GOPHER, HTTP(S) servers
cvsps-2.1_2Create patchset information from CVS
dtc-1.4.2_1Device Tree Compiler
expat-2.2.0_1  XML 1.0 parser written in C
gcc-5.4.0  GNU Compiler Collection 5
gcc-ecj-4.5Eclipse Java Compiler used to build GCC Java
gettext-runtime-0.19.8.1_1 GNU gettext runtime libraries and programs
git-2.13.0 Distributed source code management tool
glib-2.46.2_5  Some useful routines of C programming (current 
stable version)
gmake-4.2.1_1  GNU version of 'make' utility
gmp-6.1.2  Free library for arbitrary precision arithmetic
indexinfo-0.2.6Utility to regenerate the GNU info page index
libffi-3.2.1   Foreign Function Interface
libiconv-1.14_10   Character set conversion library
libnghttp2-1.21.0  HTTP/2.0 C Library
m4-1.4.18,1GNU M4
mpc-1.0.3  Library of complex numbers with arbitrarily high 
precision
mpfr-3.1.5_1   Library for multiple-precision floating-point 
computations
p5-Authen-SASL-2.16_1  Perl5 module for SASL authentication
p5-Digest-HMAC-1.03_1  Perl5 interface to HMAC Message-Digest Algorithms
p5-Error-0.17024   Error/exception handling in object-oriented 
programming style
p5-GSSAPI-0.28_1   Perl extension providing access to the GSSAPIv2 
library
pcre-8.40  Perl Compatible Regular Expressions library
perl5-5.24.1   Practical Extraction and Report Language
pixman-0.34.0  Low-level pixel manipulation library
pkg-1.10.1 Package manager
pkgconf-1.3.0,1Utility to help to configure compiler and linker 
flags
python-2.7_3,2 "meta-port" for the default version of Python 
interpreter
python2-2_3The "meta-port" for version 2 of the Python 
interpreter
python27-2.7.13_3  Interpreted object-oriented programming language
python3-3_3The "meta-port" for version 3 of the Python 
interpreter
python35-3.5.3_1   Interpreted object-oriented programming language
readline-6.3.8 Library for editing command lines as they are 
typed
sudo-1.8.20p1  Allow others to run commands as root
=== TEST BEGIN ===
Using CC: /usr/local/bin/ccache
Install prefix/var/tmp/patchew-tester-tmp-eswc2rbf/src/install
BIOS directory/var/tmp/patchew-tester-tmp-eswc2rbf/src/install/share/qemu
binary directory  /var/tmp/patchew-tester-tmp-eswc2rbf/src/install/bin
library directory /var/tmp/patchew-tester-tmp-eswc2rbf/src/install/lib
modu

[Qemu-devel] [PATCH] target-i386 : fix a race condition result to lost INIT

2017-07-30 Thread Peng Hao
when SMP VM start, AP may lost INIT because of receiving INIT between
kvm_vcpu_ioctl_x86_get/set_vcpu_events.

   vcpu 0 vcpu 1
   kvm_vcpu_ioctl_x86_get_vcpu_events
   events->smi.latched_init=0
 send INIT to vcpu1
   set vcpu1's pending_events
   kvm_vcpu_ioctl_x86_set_vcpu_events
events->smi.latched_init == 0
  clear INIT in pending_events
considering migration, just at level >= KVM_PUT_RESET_STATE,
add KVM_VCPUEVENT_VALID_SMM in events.flags.I think it is better to
modify in qemu.

Signed-off-by: Peng Hao 
---
 target/i386/kvm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index a6613e1..be146cb 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -2438,7 +2438,7 @@ static int kvm_put_vcpu_events(X86CPU *cpu, int level)
 /* Stop SMI delivery on old machine types to avoid a reboot
  * on an inward migration of an old VM.
  */
-if (!cpu->kvm_no_smi_migration) {
+if (!cpu->kvm_no_smi_migration && (level >= KVM_PUT_RESET_STATE)) {
 events.flags |= KVM_VCPUEVENT_VALID_SMM;
 }
 }
-- 
1.8.3.1





[Qemu-devel] [PATCH v7 5/5] hypertrace: Add guest-side user-level library

2017-07-30 Thread Lluís Vilanova
Provides guest library "libqemu-hypertrace-guest.a" to abstract access
to the hypertrace channel.

Signed-off-by: Lluís Vilanova 
---
 Makefile   |5 +
 configure  |2 
 hypertrace/guest/Makefile  |   30 
 hypertrace/guest/common.c  |  301 
 hypertrace/guest/qemu-hypertrace.h |   80 ++
 5 files changed, 418 insertions(+)
 create mode 100644 hypertrace/guest/Makefile
 create mode 100644 hypertrace/guest/common.c
 create mode 100644 hypertrace/guest/qemu-hypertrace.h

diff --git a/Makefile b/Makefile
index f08f498f79..523885a0a5 100644
--- a/Makefile
+++ b/Makefile
@@ -602,8 +602,13 @@ ifneq (,$(findstring qemu-ga,$(TOOLS)))
 endif
 endif
 
+install-hypertrace:
+   $(INSTALL_DIR) "$(DESTDIR)$(includedir)"
+   $(INSTALL_DATA) "$(SRC_PATH)/hypertrace/guest/qemu-hypertrace.h" 
"$(DESTDIR)$(includedir)/"
+
 
 install: all $(if $(BUILD_DOCS),install-doc) install-datadir 
install-localstatedir
+install: install-hypertrace
 ifneq ($(TOOLS),)
$(call install-prog,$(subst 
qemu-ga,qemu-ga$(EXESUF),$(TOOLS)),$(DESTDIR)$(bindir))
 endif
diff --git a/configure b/configure
index e6f752c242..9c7358d0b4 100755
--- a/configure
+++ b/configure
@@ -6286,6 +6286,8 @@ if [ "$TARGET_BASE_ARCH" = "" ]; then
 fi
 
 symlink "$source_path/Makefile.target" "$target_dir/Makefile"
+mkdir -p $target_dir/hypertrace/guest
+symlink $source_path/hypertrace/guest/Makefile 
$target_dir/hypertrace/guest/Makefile
 
 upper() {
 echo "$@"| LC_ALL=C tr '[a-z]' '[A-Z]'
diff --git a/hypertrace/guest/Makefile b/hypertrace/guest/Makefile
new file mode 100644
index 00..cbc956bde1
--- /dev/null
+++ b/hypertrace/guest/Makefile
@@ -0,0 +1,30 @@
+include ../../../config-host.mak
+include ../../config-target.mak
+include $(SRC_PATH)/rules.mak
+
+vpath % $(SRC_PATH)/hypertrace/guest
+
+# do not use QEMU's per-host cflags when building guest code
+QEMU_CFLAGS  = -Werror -Wall
+
+QEMU_CFLAGS += $(GLIB_CFLAGS)
+QEMU_CFLAGS += -I$(SRC_PATH)/include
+QEMU_CFLAGS += -I../../../linux-headers
+QEMU_CFLAGS += -I../../../
+QEMU_CFLAGS += -I../../
+
+ifdef CONFIG_SOFTMMU
+QEMU_CFLAGS += -DNEED_CPU_H
+QEMU_CFLAGS += -I$(SRC_PATH)/target-$(TARGET_BASE_ARCH)
+endif
+
+QEMU_CFLAGS += -g -O2
+
+obj-y = common.o
+
+libqemu-hypertrace-guest.a: $(obj-y)
+
+all: libqemu-hypertrace-guest.a
+
+clean:
+   rm -f $(obj-y) libqemu-hypertrace-guest.a
diff --git a/hypertrace/guest/common.c b/hypertrace/guest/common.c
new file mode 100644
index 00..96ae4464bc
--- /dev/null
+++ b/hypertrace/guest/common.c
@@ -0,0 +1,301 @@
+/*
+ * Guest-side management of hypertrace.
+ *
+ * Copyright (C) 2016-2017 Lluís Vilanova 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu-hypertrace.h"
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "config-host.h"
+#include "config-target.h"
+#if defined(CONFIG_SOFTMMU)
+#include "qemu/osdep.h"
+#include "hw/pci/pci.h"
+#endif
+#include "hypertrace/common.h"
+
+static pthread_mutex_t init_mutex = PTHREAD_MUTEX_INITIALIZER;
+
+static char *config_path;
+static int config_fd = -1;
+static uint64_t *config_addr;
+static struct hypertrace_config *config;
+
+static char *data_path;
+static int data_fd = -1;
+static uint64_t *data_addr;
+
+static char *control_path;
+static int control_fd = -1;
+#if defined(CONFIG_USER_ONLY)
+static __thread uint64_t *control_addr;
+static __thread uint64_t *control_addr_1;
+#else
+static uint64_t *control_addr;
+#endif
+
+static int page_size;
+
+
+static int init_channel_file(const char *base, const char *suffix, size_t size,
+ char **path, int *fd, uint64_t **addr, bool write)
+{
+int prot;
+
+*path = malloc(strlen(base) + strlen(suffix) + 1);
+sprintf(*path, "%s%s", base, suffix);
+
+prot = O_RDONLY;
+if (write) {
+prot = O_RDWR;
+}
+*fd = open(*path, prot);
+if (*fd == -1) {
+return -1;
+}
+
+prot = PROT_READ;
+if (write) {
+prot |= PROT_WRITE;
+}
+*addr = mmap(NULL, size, prot, MAP_SHARED, *fd, 0);
+if (*addr == MAP_FAILED) {
+return -1;
+}
+return 0;
+}
+
+#if !defined(CONFIG_USER_ONLY) && defined(__linux__)
+static int check_device_id(const char *base, const char *name, uint64_t value)
+{
+char tmp[1024];
+sprintf(tmp, "%s/%s", base, name);
+
+int fd = open(tmp, O_RDONLY);
+if (fd < 0) {
+return -1;
+}
+
+char v[1024];
+ssize_t s = read(fd, v, sizeof(v));
+if (s < 0) {
+close(fd);
+return -1;
+}
+v[s] = '\0';
+
+char *res;
+uint64_t vv = strtoull(v, &res, 16);
+if (*res == '\n' && vv == value) {
+return 0;
+} else {
+return -1;
+}
+}
+
+static char *f

[Qemu-devel] [PATCH v7 4/5] hypertrace: [softmmu] Add QEMU-side proxy to "guest_hypertrace" event

2017-07-30 Thread Lluís Vilanova
Uses a virtual device to trigger the hypertrace channel event.

Signed-off-by: Lluís Vilanova 
---
 hypertrace/Makefile.objs |4 +
 hypertrace/softmmu.c |  237 ++
 include/hw/pci/pci.h |2 
 3 files changed, 243 insertions(+)
 create mode 100644 hypertrace/softmmu.c

diff --git a/hypertrace/Makefile.objs b/hypertrace/Makefile.objs
index 2c27ad819a..1e5b8d0be2 100644
--- a/hypertrace/Makefile.objs
+++ b/hypertrace/Makefile.objs
@@ -1,10 +1,14 @@
 # -*- mode: makefile -*-
 
 target-obj-$(CONFIG_USER_ONLY) += user.o
+ifdef CONFIG_PCI
+target-obj-$(CONFIG_SOFTMMU) += softmmu.o
+endif
 target-obj-y += common.o
 
 $(obj)/user.o: $(obj)/emit.c
 $(obj)/common.o: $(obj)/emit.c
+$(obj)/softmmu.o: $(obj)/emit.c
 
 $(obj)/emit.c: $(obj)/emit.c-timestamp $(BUILD_DIR)/config-host.mak
@cmp $< $@ >/dev/null 2>&1 || cp $< $@
diff --git a/hypertrace/softmmu.c b/hypertrace/softmmu.c
new file mode 100644
index 00..4048c7e0ef
--- /dev/null
+++ b/hypertrace/softmmu.c
@@ -0,0 +1,237 @@
+/*
+ * QEMU-side management of hypertrace in softmmu emulation.
+ *
+ * Copyright (C) 2016-2017 Lluís Vilanova 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+/*
+ * Implementation details
+ * ==
+ *
+ * There are 3 channels, each a BAR of a virtual device that can be used 
through
+ * MMIO.
+ *
+ *
+ * - Configuration channel: Exposes configuration parameters.
+ *
+ * - Data channel: Lets guests write argument values. Each guest client should
+ *   use a different offset to avoid concurrency problems.
+ *
+ * - Control channel: Triggers the hypertrace event on a write, providing the
+ *   first argument. Offset in the control channel sets the offset in the data
+ *   channel.
+ */
+
+#include "qemu/osdep.h"
+#include "cpu.h"
+#include "hypertrace/common.h"
+#include "hypertrace/trace.h"
+#include "hw/pci/pci.h"
+#include "migration/blocker.h"
+#include "qapi/error.h"
+#include "qemu/error-report.h"
+
+
+typedef struct HypertraceState {
+PCIDevice dev;
+
+uint64_t max_clients;
+struct hypertrace_config hconfig;
+
+MemoryRegion config;
+void *config_ptr;
+MemoryRegion data;
+void *data_ptr;
+MemoryRegion control;
+void *control_ptr;
+
+Error *migration_blocker;
+} HypertraceState;
+
+
+static uint64_t hypertrace_control_io_read(void *opaque, hwaddr addr,
+   unsigned size)
+{
+uint64_t res;
+HypertraceState *s = opaque;
+char *mem = &((char *)s->control_ptr)[addr];
+
+switch (size) {
+case 1:
+{
+res = ((uint8_t *)mem)[0];
+break;
+}
+case 2:
+{
+res = ((uint16_t *)mem)[0];
+break;
+}
+case 4:
+{
+res = ((uint32_t *)mem)[0];
+break;
+}
+case 8:
+{
+res = ((uint64_t *)mem)[0];
+break;
+}
+default:
+error_report("error: hypertrace: Unexpected read of size %d", size);
+abort();
+}
+
+return res;
+}
+
+static void hypertrace_control_io_write(void *opaque, hwaddr addr,
+uint64_t data, unsigned size)
+{
+HypertraceState *s = opaque;
+char *mem = &((char *)s->control_ptr)[addr];
+
+switch (size) {
+case 1:
+{
+uint8_t *res = (uint8_t *)mem;
+*res = (uint8_t)data;
+break;
+}
+case 2:
+{
+uint16_t *res = (uint16_t *)mem;
+*res = (uint16_t)data;
+break;
+}
+case 4:
+{
+uint32_t *res = (uint32_t *)mem;
+*res = (uint32_t)data;
+break;
+}
+case 8:
+{
+uint64_t *res = (uint64_t *)mem;
+*res = (uint64_t)data;
+break;
+}
+default:
+error_report("error: hypertrace: Unexpected write of size %d", size);
+abort();
+}
+
+if ((addr + size) % sizeof(uint64_t) == 0) {
+uint64_t client = addr / sizeof(uint64_t);
+uint64_t vcontrol = ((uint64_t *)s->control_ptr)[client];
+uint64_t *data_ptr = (uint64_t *)s->data_ptr;
+data_ptr = &data_ptr[client * s->hconfig.client_data_size];
+hypertrace_emit(current_cpu, vcontrol, data_ptr);
+}
+}
+
+static const MemoryRegionOps hypertrace_control_ops = {
+.read = &hypertrace_control_io_read,
+.write = &hypertrace_control_io_write,
+.endianness = DEVICE_NATIVE_ENDIAN,
+.impl = {
+.min_access_size = 1,
+.max_access_size = 8,
+},
+};
+
+
+static void hypertrace_realize(PCIDevice *dev, Error **errp)
+{
+Error *local_err = NULL;
+struct hypertrace_config *config;
+HypertraceState *s = DO_UPCAST(HypertraceState, dev, dev);
+Error *err = NULL;
+
+if (s->max_clients < 1) {
+error_setg(errp, "hypertrace: must have at least one client");
+return;
+}
+
+hypertrace_init_config(&s->hconfig,

[Qemu-devel] [PATCH v7 3/5] hypertrace: [*-user] Add QEMU-side proxy to "guest_hypertrace" event

2017-07-30 Thread Lluís Vilanova
QEMU detects when the guest uses 'mmap' on hypertrace's control channel
file, and then uses 'mprotect' to detect accesses to it, which are used
to trigger tracing event "guest_hypertrace".

Signed-off-by: Lluís Vilanova 
---
 Makefile.objs|4 
 bsd-user/main.c  |   17 ++
 bsd-user/mmap.c  |   15 ++
 bsd-user/syscall.c   |   34 ++--
 hypertrace/Makefile.objs |   19 ++
 hypertrace/common.c  |   55 ++
 hypertrace/common.h  |   25 +++
 hypertrace/user.c|  415 ++
 hypertrace/user.h|   71 
 include/qom/cpu.h|4 
 linux-user/main.c|   19 ++
 linux-user/mmap.c|   16 ++
 linux-user/qemu.h|3 
 linux-user/signal.c  |   12 +
 linux-user/syscall.c |   31 ++-
 15 files changed, 714 insertions(+), 26 deletions(-)
 create mode 100644 hypertrace/Makefile.objs
 create mode 100644 hypertrace/common.c
 create mode 100644 hypertrace/common.h
 create mode 100644 hypertrace/user.c
 create mode 100644 hypertrace/user.h

diff --git a/Makefile.objs b/Makefile.objs
index ce9a60137b..57479fa738 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -98,6 +98,10 @@ util-obj-y +=  trace/
 target-obj-y += trace/
 
 ##
+# hypertrace
+target-obj-y += hypertrace/
+
+##
 # guest agent
 
 # FIXME: a few definitions from qapi-types.o/qapi-visit.o are needed
diff --git a/bsd-user/main.c b/bsd-user/main.c
index fa9c012c9f..e6bdce073f 100644
--- a/bsd-user/main.c
+++ b/bsd-user/main.c
@@ -30,9 +30,12 @@
 #include "tcg.h"
 #include "qemu/timer.h"
 #include "qemu/envlist.h"
+#include "qemu/error-report.h"
 #include "exec/log.h"
 #include "trace/control.h"
 #include "glib-compat.h"
+#include "hypertrace/user.h"
+
 
 int singlestep;
 unsigned long mmap_min_addr;
@@ -675,6 +678,8 @@ static void usage(void)
"-strace   log system calls\n"
"-trace
[[enable=]][,events=][,file=]\n"
"  specify tracing options\n"
+   "-hypertrace   [[base=]][,max-clients=]\n"
+   "  specify hypertrace options\n"
"\n"
"Environment variables:\n"
"QEMU_STRACE   Print system calls and arguments similar to 
the\n"
@@ -735,6 +740,8 @@ int main(int argc, char **argv)
 envlist_t *envlist = NULL;
 char *trace_file = NULL;
 bsd_type = target_openbsd;
+char *hypertrace_base = NULL;
+unsigned int hypertrace_max_clients = 0;
 
 if (argc <= 1)
 usage();
@@ -753,6 +760,7 @@ int main(int argc, char **argv)
 cpu_model = NULL;
 
 qemu_add_opts(&qemu_trace_opts);
+qemu_add_opts(&qemu_hypertrace_opts);
 
 optind = 1;
 for (;;) {
@@ -840,6 +848,10 @@ int main(int argc, char **argv)
 } else if (!strcmp(r, "trace")) {
 g_free(trace_file);
 trace_file = trace_opt_parse(optarg);
+} else if (!strcmp(r, "hypertrace")) {
+g_free(hypertrace_base);
+hypertrace_opt_parse(optarg, &hypertrace_base,
+ &hypertrace_max_clients);
 } else {
 usage();
 }
@@ -974,6 +986,11 @@ int main(int argc, char **argv)
 target_set_brk(info->brk);
 syscall_init();
 signal_init();
+if (atexit(hypertrace_fini) != 0) {
+error_report("error: atexit: %s", strerror(errno));
+abort();
+}
+hypertrace_init(hypertrace_base, hypertrace_size);
 
 /* Now that we've loaded the binary, GUEST_BASE is fixed.  Delay
generating the prologue until now so that the prologue can take
diff --git a/bsd-user/mmap.c b/bsd-user/mmap.c
index 7f2018ede0..6a549a3553 100644
--- a/bsd-user/mmap.c
+++ b/bsd-user/mmap.c
@@ -21,6 +21,7 @@
 #include "qemu.h"
 #include "qemu-common.h"
 #include "bsd-mman.h"
+#include "hypertrace/user.h"
 
 //#define DEBUG_MMAP
 
@@ -240,10 +241,17 @@ static abi_ulong mmap_find_vma(abi_ulong start, abi_ulong 
size)
 return addr;
 }
 
-/* NOTE: all the constants are the HOST ones */
 abi_long target_mmap(abi_ulong start, abi_ulong len, int prot,
  int flags, int fd, abi_ulong offset)
 {
+return target_mmap_cpu(start, len, prot, flags, fd, offset, NULL);
+}
+
+/* NOTE: all the constants are the HOST ones */
+abi_long target_mmap_cpu(abi_ulong start, abi_ulong len, int prot,
+ int flags, int fd, abi_ulong offset,
+ CPUState *cpu)
+{
 abi_ulong ret, end, real_start, real_end, retaddr, host_offset, host_len;
 unsigned long host_start;
 
@@ -285,6 +293,10 @@ abi_long target_mmap(abi_ulong start, abi_ulong len, int 
prot,
 goto the_end;
 real_start = start & qemu_host_page_mask;
 
+if (!hypertrace_guest_mmap_check(fd, len, offset)) {
+goto fail;
+}
+
 if (!(flags & MA

[Qemu-devel] [PATCH v7 2/5] hypertrace: Add tracing event "guest_hypertrace"

2017-07-30 Thread Lluís Vilanova
Generates the "guest_hypertrace" event with a user-configurable number
of arguments.

Signed-off-by: Lluís Vilanova 
---
 Makefile  |6 --
 Makefile.objs |2 ++
 configure |   34 ++
 rules.mak |2 +-
 trace-events  |   11 +++
 5 files changed, 52 insertions(+), 3 deletions(-)

diff --git a/Makefile b/Makefile
index 38814f9a61..f08f498f79 100644
--- a/Makefile
+++ b/Makefile
@@ -84,6 +84,7 @@ endif
 GENERATED_FILES += $(TRACE_HEADERS)
 GENERATED_FILES += $(TRACE_SOURCES)
 GENERATED_FILES += $(BUILD_DIR)/trace-events-all
+GENERATED_FILES += $(BUILD_DIR)/hypertrace/trace.h
 
 trace-group-name = $(shell dirname $1 | sed -e 's/[^a-zA-Z0-9]/_/g')
 
@@ -92,7 +93,7 @@ tracetool-y += $(shell find $(SRC_PATH)/scripts/tracetool 
-name "*.py")
 
 %/trace.h: %/trace.h-timestamp
@cmp $< $@ >/dev/null 2>&1 || cp $< $@
-%/trace.h-timestamp: $(SRC_PATH)/%/trace-events $(tracetool-y)
+%/trace.h-timestamp: %/trace-events $(tracetool-y)
$(call quiet-command,$(TRACETOOL) \
--group=$(call trace-group-name,$@) \
--format=h \
@@ -101,7 +102,7 @@ tracetool-y += $(shell find $(SRC_PATH)/scripts/tracetool 
-name "*.py")
 
 %/trace.c: %/trace.c-timestamp
@cmp $< $@ >/dev/null 2>&1 || cp $< $@
-%/trace.c-timestamp: $(SRC_PATH)/%/trace-events $(tracetool-y)
+%/trace.c-timestamp: %/trace-events $(tracetool-y)
$(call quiet-command,$(TRACETOOL) \
--group=$(call trace-group-name,$@) \
--format=c \
@@ -534,6 +535,7 @@ distclean: clean
rm -Rf .sdk
if test -f pixman/config.log; then $(MAKE) -C pixman distclean; fi
if test -f dtc/version_gen.h; then $(MAKE) $(DTC_MAKE_ARGS) clean; fi
+   rm -f hypertrace/trace-events
 
 KEYMAPS=da en-gb  et  fr fr-ch  is  lt  modifiers  no  pt-br  sv \
 ar  de en-us  fi  fr-be  hr it  lv  nl pl  ru th \
diff --git a/Makefile.objs b/Makefile.objs
index bfd5a6ceb1..ce9a60137b 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -170,9 +170,11 @@ trace-events-subdirs += accel/kvm
 trace-events-subdirs += nbd
 
 trace-events-files = $(SRC_PATH)/trace-events 
$(trace-events-subdirs:%=$(SRC_PATH)/%/trace-events)
+trace-events-files += $(BUILD_DIR)/hypertrace/trace-events
 
 trace-obj-y = trace-root.o
 trace-obj-y += $(trace-events-subdirs:%=%/trace.o)
 trace-obj-$(CONFIG_TRACE_UST) += trace-ust-all.o
 trace-obj-$(CONFIG_TRACE_DTRACE) += trace-dtrace-root.o
 trace-obj-$(CONFIG_TRACE_DTRACE) += $(trace-events-subdirs:%=%/trace-dtrace.o)
+trace-obj-y += $(BUILD_DIR)/hypertrace/trace.o
diff --git a/configure b/configure
index a3f0522e8f..e6f752c242 100755
--- a/configure
+++ b/configure
@@ -351,6 +351,7 @@ pie=""
 qom_cast_debug="yes"
 trace_backends="log"
 trace_file="trace"
+hypertrace="1"
 spice=""
 rbd=""
 smartcard=""
@@ -895,6 +896,8 @@ for opt do
   ;;
   --with-trace-file=*) trace_file="$optarg"
   ;;
+  --with-hypertrace-args=*) hypertrace="$optarg"
+  ;;
   --enable-gprof) gprof="yes"
   ;;
   --enable-gcov) gcov="yes"
@@ -1425,6 +1428,8 @@ Advanced options (experts only):
Available backends: $trace_backend_list
   --with-trace-file=NAME   Full PATH,NAME of file to store traces
Default:trace-
+  --with-hypertrace-args=NUMBER
+   number of hypertrace arguments (default: 
$hypertrace)
   --disable-slirp  disable SLIRP userspace network connectivity
   --enable-tcg-interpreter enable TCG with bytecode interpreter (TCI)
   --oss-libpath to OSS library
@@ -4434,6 +4439,16 @@ if test "$?" -ne 0 ; then
 fi
 
 ##
+# check hypertrace arguments
+case "$hypertrace" in
+''|*[!0-9]*) error_exit "invalid number of hypertrace arguments" ;;
+*) ;;
+esac
+if test $hypertrace = 0; then
+error_exit "hypertrace arguments must be greater than zero"
+fi
+
+##
 # For 'ust' backend, test if ust headers are present
 if have_backend "ust"; then
   cat > $TMPC << EOF
@@ -5282,6 +5297,7 @@ echo "Trace backends$trace_backends"
 if have_backend "simple"; then
 echo "Trace output file $trace_file-"
 fi
+echo "Hypertrace arguments  $hypertrace"
 echo "spice support $spice $(echo_version $spice 
$spice_protocol_version/$spice_server_version)"
 echo "rbd support   $rbd"
 echo "xfsctl support$xfs"
@@ -5989,6 +6005,24 @@ else
 fi
 QEMU_INCLUDES="-I\$(SRC_PATH)/tcg $QEMU_INCLUDES"
 
+# hypertrace
+echo "CONFIG_HYPERTRACE_ARGS=$hypertrace" >> $config_host_mak
+hypertrace_events=hypertrace/trace-events
+mkdir -p $(dirname $hypertrace_events)
+echo "# See docs/trace-events.txt for syntax documentation." 
>$hypertrace_events
+echo -n 'vcpu guest_hypertrace(' >>$hypertrace_events
+for i in `seq $hypertrace`; do
+if test $i != 1; then
+echo -n ", " >>$hypertrace_events
+fi
+echo -n "uint64_t arg$i" >>$hypertrac

[Qemu-devel] [PATCH v7 1/5] hypertrace: Add documentation

2017-07-30 Thread Lluís Vilanova
Signed-off-by: Lluís Vilanova 
---
 docs/devel/tracing.txt |3 +
 docs/hypertrace.txt|  225 
 2 files changed, 228 insertions(+)
 create mode 100644 docs/hypertrace.txt

diff --git a/docs/devel/tracing.txt b/docs/devel/tracing.txt
index 5768a0b7a2..9178a308da 100644
--- a/docs/devel/tracing.txt
+++ b/docs/devel/tracing.txt
@@ -5,6 +5,9 @@
 This document describes the tracing infrastructure in QEMU and how to use it
 for debugging, profiling, and observing execution.
 
+See "docs/hypertrace.txt" to correlate guest tracing events with those in the
+QEMU host.
+
 == Quickstart ==
 
 1. Build with the 'simple' trace backend:
diff --git a/docs/hypertrace.txt b/docs/hypertrace.txt
new file mode 100644
index 00..c3715db25b
--- /dev/null
+++ b/docs/hypertrace.txt
@@ -0,0 +1,225 @@
+= Hypertrace channel =
+
+Copyright (C) 2016-2017 Lluís Vilanova 
+
+This work is licensed under the terms of the GNU GPL, version 2 or later.
+See the COPYING file in the top-level directory.
+
+
+The hypertrace channel allows guest code to emit events in QEMU (the host) 
using
+its tracing infrastructure (see "docs/trace.txt"). This works in both 'system'
+and 'user' modes. Therefore, hypertrace is to tracing what hypercalls are to
+system calls.
+
+The hypertrace channel can be used for various purposes:
+
+* Using guest code semantics to guide which QEMU events to trace at each point
+  in time. The example "Quick guide" below shows how to use this to identify
+  "regions of interest" in your guest code. It then uses these regions to trace
+  QEMU's behaviour during their execution, without paying the price of tracing
+  events outside the interest regions.
+
+* Mark "progress points" in guest code (e.g., processed client requests,
+  scheduled processes, etc), so that they can be easily traced and correlated
+  between QEMU's various tracing events and the guest's own tracing
+  infrastructure (e.g., Linux's tracepoints).
+
+* You can also use regions of interest and progress points on the guest code to
+  time the performance of new TCG optimizations. Each hypertrace event comes
+  with a host timestamp, making it easy to compare the host execution times of
+  interesting guest code.
+
+Hypertrace features:
+
+* Works with 'system' and 'user' mode.
+
+* Minimal setup for the guest; QEMU provides support guest code libraries that
+  work out of the box.
+
+* Independent of guest architecture; the guest code uses accesses to special
+  memory regions, as opposed to redefining instruction semantics.
+
+* Negligible guest overhead; emitting a hypertrace event requires a single 
guest
+  memory access, making it as unobtrusive as possible.
+
+Warning: The hypertrace channel in 'system' mode only works in systems with
+support for PCI. You can get the list of guests with PCI support with 'grep
+pci.mak default-configs/*'.
+
+
+== Quick guide ==
+
+This shows an example of using the hypertrace channel to trace the guest memory
+accesses only in a specific guest code region, which is identified by calls to
+the hypertrace channel.
+
+We are going to trace memory accesses to disk using QEMU's "log" backend, and
+will use QEMU's "dtrace" backend (SystemTap) to ensure memory accesses are only
+traced in the guest code region of interest. The first time the guest code
+invokes the hypertrace channel, we will start tracing the
+"guest_mem_before_exec" event using dtrace, and then will disable it the second
+time around.
+
+Tracing is done with "log" because it is more efficient than using "dtrace" in
+high-volume events like memory accesses.
+
+1. Set the tracing backends and number of arguments for the hypertrace events:
+
+mkdir /tmp/qemu-build
+cd /tmp/qemu-build
+/path/to/qemu-source/configure  \
+--enable-trace-backends=dtrace,log  \
+--with-hypertrace-args=4\
+--prefix=/tmp/qemu-install
+make -j install
+
+2. Compile QEMU:
+
+make -C /tmp/qemu-build install -j
+
+3. Compile the guest support code:
+
+make -C /tmp/qemu-build/x86_64-linux-user/hypertrace/guest
+make -C /tmp/qemu-build/x86_64-softmmu/hypertrace/guest
+
+   If you need to cross-compile the guest library, set the 'CC' variable:
+
+make -C /tmp/qemu-build/mipsel-linux-user/hypertrace/guest 
CC=mipsel-gnu-linux-gcc
+
+4. Create a guest application that interacts with the hypertrace channel:
+
+cat > /tmp/my-hypertrace.c <<\EOF
+#include 
+#include 
+#include 
+#include 
+#include 
+
+
+int main(int argc, char **argv)
+{
+char *base = NULL;
+if (argc > 1) {
+base = argv[1];
+}
+
+/* In 'user' mode this path must be the same we will use to start 
QEMU. */
+if (qemu_hypertrace_init(base) != 0) {
+perror("error: qemu_hypertrace_init");
+abort();
+}
+
+/* Set additional event arguments (unused in 

[Qemu-devel] [PATCH v7 0/5] hypertrace: Lightweight guest-to-QEMU trace channel

2017-07-30 Thread Lluís Vilanova
The hypertrace channel allows guest code to emit events in QEMU (the host) using
its tracing infrastructure (see "docs/trace.txt"). This works in both 'system'
and 'user' modes, is architecture-agnostic and introduces minimal noise on the
guest.

See first commit for a full description, use-cases and an example.

Signed-off-by: Lluís Vilanova 
---

Changes in v7
=

* Use 'expr' instead of assuming 'bash' when generating the "emit.c" file.
* Restore generation of trace-events-all.


Changes in v6
=

* Fix compilation errors.


Changes in v5
=

* Rebase on 5a477a7806.
* Fix typo in "bsd-user/main.c" [Stephan Hajnoczi].
* Replace abort() with exit() in command-line errors [Stephan Hajnoczi].
* Fix alignment of data and control channels [Stephan Hajnoczi].
* Fix signal reflection in user-mode (SIGINT, SIGABRT, SIGSEGV) [Stephan 
Hajnoczi].
* Clarify semantics of hypertrace_guest_mmap_check() [Stephan Hajnoczi].
* Use uintptr_t instead of unsigned long in SEGV handler [Stephan Hajnoczi].
* Emit hypertrace's event with host-endian arguments [Stephan Hajnoczi].
* Enable true concurrency between user-mode guest threads by using a spearate 
control channel page per client [Stephan Hajnoczi].
* Remove unused PAGE_SIZE define [Stephan Hajnoczi].
* Submit linux kernel API module separately to Linux upstream [Stephan 
Hajnoczi].
* Assume guest code events are always enabled.


Changes in v4
=

* Fix typo in stap script example.
* Fix compilation instructions in doc/hypertrace.txt.
* Rebase on 0737f32daf.


Changes in v3
=

* Rebase on 4a58f35.
* Remove debugging printf's.
* Fix style issues identified by checkpatch.
* Fix control channel mapping in guest linux module.
* Add a short event description in "trace-events".
* Polish documentation in 1st patch.


Changes in v2
=

* Remove unnecessary casts for g2h() [Eric Blake].
* Use perror() [Eric Blake].
* Avoid expansions in application example [Eric Blake].
* Add copyright in document "hypertrace.txt" [Eric Blake].
* Make the user-mode hypertrace invocations thread-safe [Stefan Hajnoczi].
* Split dynamic hypertrace configuration into a separate "config" channel.

Lluís Vilanova (5):
  hypertrace: Add documentation
  hypertrace: Add tracing event "guest_hypertrace"
  hypertrace: [*-user] Add QEMU-side proxy to "guest_hypertrace" event
  hypertrace: [softmmu] Add QEMU-side proxy to "guest_hypertrace" event
  hypertrace: Add guest-side user-level library


 Makefile   |   11 +
 Makefile.objs  |6 +
 bsd-user/main.c|   17 +
 bsd-user/mmap.c|   15 +
 bsd-user/syscall.c |   34 ++-
 configure  |   36 +++
 docs/devel/tracing.txt |3 
 docs/hypertrace.txt|  225 
 hypertrace/Makefile.objs   |   23 ++
 hypertrace/common.c|   55 +
 hypertrace/common.h|   25 ++
 hypertrace/guest/Makefile  |   30 +++
 hypertrace/guest/common.c  |  301 ++
 hypertrace/guest/qemu-hypertrace.h |   80 +++
 hypertrace/softmmu.c   |  237 +
 hypertrace/user.c  |  415 
 hypertrace/user.h  |   71 ++
 include/hw/pci/pci.h   |2 
 include/qom/cpu.h  |4 
 linux-user/main.c  |   19 ++
 linux-user/mmap.c  |   16 +
 linux-user/qemu.h  |3 
 linux-user/signal.c|   12 +
 linux-user/syscall.c   |   31 ++-
 rules.mak  |2 
 trace-events   |   11 +
 26 files changed, 1655 insertions(+), 29 deletions(-)
 create mode 100644 docs/hypertrace.txt
 create mode 100644 hypertrace/Makefile.objs
 create mode 100644 hypertrace/common.c
 create mode 100644 hypertrace/common.h
 create mode 100644 hypertrace/guest/Makefile
 create mode 100644 hypertrace/guest/common.c
 create mode 100644 hypertrace/guest/qemu-hypertrace.h
 create mode 100644 hypertrace/softmmu.c
 create mode 100644 hypertrace/user.c
 create mode 100644 hypertrace/user.h


To: qemu-devel@nongnu.org
Cc: Stefan Hajnoczi 
Cc: Eric Blake 
Cc: Luiz Capitulino 
Cc: Daniel P Berrange