[Xen-devel] [xen-unstable-smoke test] 121369: tolerable all pass - PUSHED

[osstest service owner]

flight 121369 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/121369/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  a92fd9a4452cd82fa86ce1ecd6f02d53ec139c45
baseline version:
 xen  ebe29cbd338aba99a0e17ecbdc73a25545bd219a

Last test of basis   121346  2018-03-29 18:01:05 Z0 days
Testing same since   121348  2018-03-29 21:16:51 Z0 days3 attempts


People who touched revisions under test:
  Andre Przywara 
  Julien Grall 
  Stefano Stabellini 

jobs:
 build-arm64-xsm  pass
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   ebe29cbd33..a92fd9a445  a92fd9a4452cd82fa86ce1ecd6f02d53ec139c45 -> smoke

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] Need some advices on how to workaround a hardware bug

[Chao Gao]

Hi,

I met an EPT violation and then the guest was destroyed by Xen
after assigning a device to the guest. After some investigation, I found
it is caused by the device isn't a standard PCI device -- its MSI-x PBA
locates in the same 4k-byte page with other CSR. When the driver in
guest writes the registers in that page, an EPT violation happens because
the PBA page is marked as read-only by the below line in
msix_capability_init()
if ( rangeset_add_range(mmio_ro_ranges, msix->pba.first,
msix->pba.last) )
The reason why Xen marks this page read-only I think is PCI SPEC says:
Software should never write, and should only read Pending Bits.
If software writes to Pending Bits, the result is undefined
Then Xen goes through all registered MMIO range and finds this address
hasn't been registered. Thus it destroys the guest.

I plan to work out a workaround for this issue to allow Xen guest (also
dom0 if dom0 uses EPT? not sure) to use devices efficiently which
violate PCI SPEC in this way. Currently, there are two options (EPT SPP
might provide a perfect solution) :

One is trapping the page where PBA locates and ignoring writes to PBA and
apply writes to other fields in the same page. It would incur significant
performance degradation if this page is accessed frequently. In order
to mitigate the performance drop, a patch to trap PBA lazily like what
qemu does [1] is needed.

The other is Do not trap accesses to the page where PBA locates. In this
option, all accesses to the page will go to hardware device without
Xen's interception. I think one concern would be whether this option
would lead to bring some security holes, compared with trapping these accesses.
In my mind, the answer is no because Xen even doesn't read PBA. A corner
case for this option might be PBA resides in the same page with MSIx table,
which is allowed according to the following description in PCI SPEC:
The MSI-X Table and MSI-X PBA are permitted to co-reside within a
naturally aligned 4-KB address range, though they must not overlap with
each other.

Which one do you think is better? or any other thoughts about how to
workaround this case?

[1]:https://git.qemu.org/?p=qemu.git;a=commit;h=95239e162518dc6577164be3d9a789aba7f591a3

Thanks
Chao

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-unstable-smoke test] 121354: regressions - FAIL

[osstest service owner]

flight 121354 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/121354/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-libvirt   6 libvirt-build  fail in 121348 REGR. vs. 121346

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemuu-debianhvm-i386 16 guest-localmigrate/x10 fail pass 
in 121348

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt  1 build-check(1)   blocked in 121348 n/a
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  a92fd9a4452cd82fa86ce1ecd6f02d53ec139c45
baseline version:
 xen  ebe29cbd338aba99a0e17ecbdc73a25545bd219a

Last test of basis   121346  2018-03-29 18:01:05 Z0 days
Testing same since   121348  2018-03-29 21:16:51 Z0 days2 attempts


People who touched revisions under test:
  Andre Przywara 
  Julien Grall 
  Stefano Stabellini 

jobs:
 build-arm64-xsm  pass
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-i386 fail
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.


commit a92fd9a4452cd82fa86ce1ecd6f02d53ec139c45
Author: Andre Przywara 
Date:   Thu Aug 24 17:26:32 2017 +0100

ARM: VGIC: wire new VGIC(-v2) files into Xen build system

Now that we have both the old VGIC prepared to cope with a sibling and
the code for the new VGIC in place, lets add a Kconfig option to enable
the new code and wire it into the Xen build system.
This will add a compile time option to use either the "old" or the "new"
VGIC.
In the moment this is restricted to a vGIC-v2. To make the build system
happy, we provide a temporary dummy implementation of
vgic_v3_setup_hw() to allow building for now.

Signed-off-by: Andre Przywara 
Acked-by: Julien Grall 
Acked-by: Stefano Stabellini 

commit b77d774d8274183c2252f5fbc9fa3b3b7022ba06
Author: Andre Przywara 
Date:   Thu Dec 21 12:41:28 2017 +

ARM: new VGIC: Allocate two pages for struct vcpu

At the moment we allocate exactly one page for struct vcpu on ARM, also
have a check in place to prevent it growing beyond 4KB.
As the struct includes the state of all 32 private (per-VCPU) interrupts,
we are at 3840 bytes on arm64 at the moment already. Growing the per-IRQ
VGIC structure even slightly makes the VCPU quickly exceed the 4K limit.
The new VGIC will need more space per virtual IRQ. I spent a few hours
trying to trim this down, but couldn't get it below 4KB, even with the
nasty hacks piling up to save some bytes here and there.
It turns out that beyond efficiency, maybe, there is no real technical
reason this struct has to fit in one page, so lifting the limit to two
pages seems like the most pragmatic solution.
Restrict the compilation error to compiling with the new VGIC and for
ARM64 only.

Signed-off-by: Andre Przywara 
Acked-by: Julien Grall 
Acked-by: Stefano Stabellini 

commit be326763e8844b8f2b4213c4f5036970e538054b
Author: Andre Przywara 
Date:   Wed Feb 7 14:54:23 2018 +

ARM: new VGIC: 

[Xen-devel] [linux-4.1 test] 121332: regressions - trouble: broken/fail/pass

[osstest service owner]

flight 121332 linux-4.1 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/121332/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-libvirt-pair broken
 test-amd64-i386-libvirt-pair 5 host-install/dst_host(5) broken REGR. vs. 118294
 test-armhf-armhf-xl-credit2 16 guest-start/debian.repeat fail REGR. vs. 118294

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds 17 guest-start.2fail REGR. vs. 118294

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 118294
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 118294
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail  like 118294
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 118294
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 118294
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 118294
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop fail like 118294
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-pvhv2-intel 12 guest-start fail never pass
 test-amd64-amd64-xl-pvhv2-amd 12 guest-start  fail  never pass
 test-arm64-arm64-examine  8 reboot   fail   never pass
 test-arm64-arm64-libvirt-xsm  7 xen-boot fail   never pass
 test-arm64-arm64-xl-credit2   7 xen-boot fail   never pass
 test-arm64-arm64-xl   7 xen-boot fail   never pass
 test-arm64-arm64-xl-xsm   7 xen-boot fail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stop fail never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass

version targeted for testing:
 linux2d61e08a1024d0cf15c26889285004e46c9f0b14
baseline version:
 linux30ad2851a645bb5f42c72f21ceb166877cf7e695

Last test of basis   118294  2018-01-23 23:50:01 Z   65 days
Failing since120338  2018-03-08 06:19:32 Z   21 days   14 attempts
Testing same since   121332  2018-03-28 14:20:24 Z  

[Xen-devel] [xen-unstable-smoke test] 121348: regressions - FAIL

[osstest service owner]

flight 121348 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/121348/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-libvirt   6 libvirt-buildfail REGR. vs. 121346

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  a92fd9a4452cd82fa86ce1ecd6f02d53ec139c45
baseline version:
 xen  ebe29cbd338aba99a0e17ecbdc73a25545bd219a

Last test of basis   121346  2018-03-29 18:01:05 Z0 days
Testing same since   121348  2018-03-29 21:16:51 Z0 days1 attempts


People who touched revisions under test:
  Andre Przywara 
  Julien Grall 
  Stefano Stabellini 

jobs:
 build-arm64-xsm  pass
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  fail
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt blocked 



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.


commit a92fd9a4452cd82fa86ce1ecd6f02d53ec139c45
Author: Andre Przywara 
Date:   Thu Aug 24 17:26:32 2017 +0100

ARM: VGIC: wire new VGIC(-v2) files into Xen build system

Now that we have both the old VGIC prepared to cope with a sibling and
the code for the new VGIC in place, lets add a Kconfig option to enable
the new code and wire it into the Xen build system.
This will add a compile time option to use either the "old" or the "new"
VGIC.
In the moment this is restricted to a vGIC-v2. To make the build system
happy, we provide a temporary dummy implementation of
vgic_v3_setup_hw() to allow building for now.

Signed-off-by: Andre Przywara 
Acked-by: Julien Grall 
Acked-by: Stefano Stabellini 

commit b77d774d8274183c2252f5fbc9fa3b3b7022ba06
Author: Andre Przywara 
Date:   Thu Dec 21 12:41:28 2017 +

ARM: new VGIC: Allocate two pages for struct vcpu

At the moment we allocate exactly one page for struct vcpu on ARM, also
have a check in place to prevent it growing beyond 4KB.
As the struct includes the state of all 32 private (per-VCPU) interrupts,
we are at 3840 bytes on arm64 at the moment already. Growing the per-IRQ
VGIC structure even slightly makes the VCPU quickly exceed the 4K limit.
The new VGIC will need more space per virtual IRQ. I spent a few hours
trying to trim this down, but couldn't get it below 4KB, even with the
nasty hacks piling up to save some bytes here and there.
It turns out that beyond efficiency, maybe, there is no real technical
reason this struct has to fit in one page, so lifting the limit to two
pages seems like the most pragmatic solution.
Restrict the compilation error to compiling with the new VGIC and for
ARM64 only.

Signed-off-by: Andre Przywara 
Acked-by: Julien Grall 
Acked-by: Stefano Stabellini 

commit be326763e8844b8f2b4213c4f5036970e538054b
Author: Andre Przywara 
Date:   Wed Feb 7 14:54:23 2018 +

ARM: new VGIC: vgic-init: implement map_resources

map_resources is the last initialization step needed before the first
VCPU is run. At that stage the code stores the MMIO base addresses used.
Also it registers the respective register 

Re: [Xen-devel] [PATCH v2] xl/libxl: add pvcalls support

[Jim Fehlig]

On 03/29/2018 04:07 PM, Stefano Stabellini wrote:

Add pvcalls support to libxl and xl. Create the appropriate pvcalls
entries in xenstore.

Signed-off-by: Stefano Stabellini 

---

Changes in v2:
- rename pvcalls to pvcallsif internally in libxl to avoid `pvcallss'
---
  docs/misc/xenstore-paths.markdown|  9 +
  tools/libxl/Makefile |  2 +-
  tools/libxl/libxl.h  | 10 ++
  tools/libxl/libxl_create.c   |  4 
  tools/libxl/libxl_internal.h |  1 +
  tools/libxl/libxl_pvcalls.c  | 37 
  tools/libxl/libxl_types.idl  |  7 +++
  tools/libxl/libxl_types_internal.idl |  1 +
  tools/xl/xl_parse.c  | 37 +++-
  9 files changed, 106 insertions(+), 2 deletions(-)
  create mode 100644 tools/libxl/libxl_pvcalls.c

diff --git a/docs/misc/xenstore-paths.markdown 
b/docs/misc/xenstore-paths.markdown
index 7be2592..77d1a36 100644
--- a/docs/misc/xenstore-paths.markdown
+++ b/docs/misc/xenstore-paths.markdown
@@ -299,6 +299,11 @@ A virtual scsi device frontend. Described by
  A virtual usb device frontend. Described by
  [xen/include/public/io/usbif.h][USBIF]
  
+ ~/device/pvcalls/$DEVID/* []

+
+Paravirtualized POSIX function calls frontend. Described by
+[docs/misc/pvcalls.markdown][PVCALLS]
+
   ~/console/* []
  
  The primary PV console device. Described in [console.txt](console.txt)

@@ -377,6 +382,10 @@ A PV SCSI backend.
  
  A PV USB backend. Described by

  [xen/include/public/io/usbif.h][USBIF]
+
+ ~/backend/pvcalls/$DOMID/$DEVID/* []
+
+A PVCalls backend. Described in [docs/misc/pvcalls.markdown][PVCALLS].
  
   ~/backend/console/$DOMID/$DEVID/* []
  
diff --git a/tools/libxl/Makefile b/tools/libxl/Makefile

index 917ceb0..035e66e 100644
--- a/tools/libxl/Makefile
+++ b/tools/libxl/Makefile
@@ -140,7 +140,7 @@ LIBXL_OBJS = flexarray.o libxl.o libxl_create.o libxl_dm.o 
libxl_pci.o \
libxl_vtpm.o libxl_nic.o libxl_disk.o libxl_console.o \
libxl_cpupool.o libxl_mem.o libxl_sched.o libxl_tmem.o \
libxl_9pfs.o libxl_domain.o libxl_vdispl.o \
-$(LIBXL_OBJS-y)
+libxl_pvcalls.o $(LIBXL_OBJS-y)
  LIBXL_OBJS += libxl_genid.o
  LIBXL_OBJS += _libxl_types.o libxl_flask.o _libxl_types_internal.o
  
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h

index eca0ea2..c4eccc5 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -2006,6 +2006,16 @@ int libxl_device_p9_destroy(libxl_ctx *ctx, uint32_t 
domid,
  const libxl_asyncop_how *ao_how)
  LIBXL_EXTERNAL_CALLERS_ONLY;
  
+/* pvcalls interface */

+int libxl_device_pvcallsif_remove(libxl_ctx *ctx, uint32_t domid,
+  libxl_device_pvcallsif *pvcallsif,
+  const libxl_asyncop_how *ao_how)
+  LIBXL_EXTERNAL_CALLERS_ONLY;
+int libxl_device_pvcallsif_destroy(libxl_ctx *ctx, uint32_t domid,
+   libxl_device_pvcallsif *pvcallsif,
+   const libxl_asyncop_how *ao_how)
+   LIBXL_EXTERNAL_CALLERS_ONLY;
+
  /* PCI Passthrough */
  int libxl_device_pci_add(libxl_ctx *ctx, uint32_t domid,
   libxl_device_pci *pcidev,
diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index c498135..c43f391 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -1374,6 +1374,10 @@ static void domcreate_launch_dm(libxl__egc *egc, 
libxl__multidev *multidev,
  for (i = 0; i < d_config->num_p9s; i++)
  libxl__device_add(gc, domid, __p9_devtype, _config->p9s[i]);
  
+for (i = 0; i < d_config->num_pvcallsifs; i++)

+libxl__device_add(gc, domid, __pvcallsif_devtype,
+  _config->pvcallsifs[i]);
+
  switch (d_config->c_info.type) {
  case LIBXL_DOMAIN_TYPE_HVM:
  {
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index 506687f..50209ff 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -3648,6 +3648,7 @@ extern const struct libxl_device_type 
libxl__usbdev_devtype;
  extern const struct libxl_device_type libxl__pcidev_devtype;
  extern const struct libxl_device_type libxl__vdispl_devtype;
  extern const struct libxl_device_type libxl__p9_devtype;
+extern const struct libxl_device_type libxl__pvcallsif_devtype;
  
  extern const struct libxl_device_type *device_type_tbl[];
  
diff --git a/tools/libxl/libxl_pvcalls.c b/tools/libxl/libxl_pvcalls.c

new file mode 100644
index 000..bb6f307
--- /dev/null
+++ b/tools/libxl/libxl_pvcalls.c
@@ -0,0 +1,37 @@
+/*
+ * Copyright (C) 2018  Aporeto
+ * Author Stefano Stabellini 
+ *
+ * This 

[Xen-devel] [PATCH v2] xl/libxl: add pvcalls support

[Stefano Stabellini]

Add pvcalls support to libxl and xl. Create the appropriate pvcalls
entries in xenstore.

Signed-off-by: Stefano Stabellini 

---

Changes in v2:
- rename pvcalls to pvcallsif internally in libxl to avoid `pvcallss'
---
 docs/misc/xenstore-paths.markdown|  9 +
 tools/libxl/Makefile |  2 +-
 tools/libxl/libxl.h  | 10 ++
 tools/libxl/libxl_create.c   |  4 
 tools/libxl/libxl_internal.h |  1 +
 tools/libxl/libxl_pvcalls.c  | 37 
 tools/libxl/libxl_types.idl  |  7 +++
 tools/libxl/libxl_types_internal.idl |  1 +
 tools/xl/xl_parse.c  | 37 +++-
 9 files changed, 106 insertions(+), 2 deletions(-)
 create mode 100644 tools/libxl/libxl_pvcalls.c

diff --git a/docs/misc/xenstore-paths.markdown 
b/docs/misc/xenstore-paths.markdown
index 7be2592..77d1a36 100644
--- a/docs/misc/xenstore-paths.markdown
+++ b/docs/misc/xenstore-paths.markdown
@@ -299,6 +299,11 @@ A virtual scsi device frontend. Described by
 A virtual usb device frontend. Described by
 [xen/include/public/io/usbif.h][USBIF]
 
+ ~/device/pvcalls/$DEVID/* []
+
+Paravirtualized POSIX function calls frontend. Described by
+[docs/misc/pvcalls.markdown][PVCALLS]
+
  ~/console/* []
 
 The primary PV console device. Described in [console.txt](console.txt)
@@ -377,6 +382,10 @@ A PV SCSI backend.
 
 A PV USB backend. Described by
 [xen/include/public/io/usbif.h][USBIF]
+ 
+ ~/backend/pvcalls/$DOMID/$DEVID/* []
+
+A PVCalls backend. Described in [docs/misc/pvcalls.markdown][PVCALLS].
 
  ~/backend/console/$DOMID/$DEVID/* []
 
diff --git a/tools/libxl/Makefile b/tools/libxl/Makefile
index 917ceb0..035e66e 100644
--- a/tools/libxl/Makefile
+++ b/tools/libxl/Makefile
@@ -140,7 +140,7 @@ LIBXL_OBJS = flexarray.o libxl.o libxl_create.o libxl_dm.o 
libxl_pci.o \
libxl_vtpm.o libxl_nic.o libxl_disk.o libxl_console.o \
libxl_cpupool.o libxl_mem.o libxl_sched.o libxl_tmem.o \
libxl_9pfs.o libxl_domain.o libxl_vdispl.o \
-$(LIBXL_OBJS-y)
+libxl_pvcalls.o $(LIBXL_OBJS-y)
 LIBXL_OBJS += libxl_genid.o
 LIBXL_OBJS += _libxl_types.o libxl_flask.o _libxl_types_internal.o
 
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index eca0ea2..c4eccc5 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -2006,6 +2006,16 @@ int libxl_device_p9_destroy(libxl_ctx *ctx, uint32_t 
domid,
 const libxl_asyncop_how *ao_how)
 LIBXL_EXTERNAL_CALLERS_ONLY;
 
+/* pvcalls interface */
+int libxl_device_pvcallsif_remove(libxl_ctx *ctx, uint32_t domid,
+  libxl_device_pvcallsif *pvcallsif,
+  const libxl_asyncop_how *ao_how)
+  LIBXL_EXTERNAL_CALLERS_ONLY;
+int libxl_device_pvcallsif_destroy(libxl_ctx *ctx, uint32_t domid,
+   libxl_device_pvcallsif *pvcallsif,
+   const libxl_asyncop_how *ao_how)
+   LIBXL_EXTERNAL_CALLERS_ONLY;
+
 /* PCI Passthrough */
 int libxl_device_pci_add(libxl_ctx *ctx, uint32_t domid,
  libxl_device_pci *pcidev,
diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index c498135..c43f391 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -1374,6 +1374,10 @@ static void domcreate_launch_dm(libxl__egc *egc, 
libxl__multidev *multidev,
 for (i = 0; i < d_config->num_p9s; i++)
 libxl__device_add(gc, domid, __p9_devtype, _config->p9s[i]);
 
+for (i = 0; i < d_config->num_pvcallsifs; i++)
+libxl__device_add(gc, domid, __pvcallsif_devtype,
+  _config->pvcallsifs[i]);
+
 switch (d_config->c_info.type) {
 case LIBXL_DOMAIN_TYPE_HVM:
 {
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index 506687f..50209ff 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -3648,6 +3648,7 @@ extern const struct libxl_device_type 
libxl__usbdev_devtype;
 extern const struct libxl_device_type libxl__pcidev_devtype;
 extern const struct libxl_device_type libxl__vdispl_devtype;
 extern const struct libxl_device_type libxl__p9_devtype;
+extern const struct libxl_device_type libxl__pvcallsif_devtype;
 
 extern const struct libxl_device_type *device_type_tbl[];
 
diff --git a/tools/libxl/libxl_pvcalls.c b/tools/libxl/libxl_pvcalls.c
new file mode 100644
index 000..bb6f307
--- /dev/null
+++ b/tools/libxl/libxl_pvcalls.c
@@ -0,0 +1,37 @@
+/*
+ * Copyright (C) 2018  Aporeto
+ * Author Stefano Stabellini 
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser 

Re: [Xen-devel] [PATCH] xl/libxl: add pvcalls support

[Stefano Stabellini]

On Thu, 29 Mar 2018, Ian Jackson wrote:
> Stefano Stabellini writes ("[PATCH] xl/libxl: add pvcalls support"):
> > Add pvcalls support to libxl and xl. Create the appropriate pvcalls
> > entries in xenstore.
> ...
> > + ~/device/pvcalls/$DEVID/* []
> > +
> > +Paravirtualized POSIX function calls frontend. Described by
> > +[docs/misc/pvcalls.markdown][PVCALLS]
> 
> It's not entirely clear what the semantics are if multiple pvcalls
> devices are provided.  Which is the guest expected to use ?
> 
> Perhaps the doc should state some convention, if there is one.  I hope
> there is such a convention ($DEVID usually 0 maybe?)

It would be similar to providing two network cards to the guest. Either
one can be used. But there is no way to provide meta-information on
which one should be used for what at the moment. As you point out, and
as per other PV protocols, the first one uses DEVID 0.


> > +for (i = 0; i < d_config->num_pvcallss; i++)
> 
> The name `pvcallss' is clumsy.  But I'm not sure I have a much better
> suggestion.
> 
> One idea might be to rename your whole thing `pvrpc' since it's a
> general RPC scheme, more or less.  Except that I don't want to come in
> now and say you should rename it.  And also most rpc systems have an
> idl language and you have ad hoc binary structs.

I don't think it is a good idea at this stage. All the documents and
presentations refer to it as "pvcalls" including
docs/misc/pvcalls.markdown.


> Have you considered calling this `pvcallsifs' ?  Where `if' is
> `intrerface' ?  Or something ?

This is a good suggestion, thank you. I'll send a v2 of the patch.


> Anyway, I would like you to think about this and answer my questions
> but I don't think it's a blocker. 

Thank you for the good feedback.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-4.7-testing test] 121330: regressions - FAIL

[osstest service owner]

flight 121330 xen-4.7-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/121330/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-xtf-amd64-amd64-3 50 xtf/test-hvm64-lbr-tsx-vmentry fail REGR. vs. 121247
 test-armhf-armhf-xl-credit2 16 guest-start/debian.repeat fail REGR. vs. 121247
 build-armhf-libvirt   6 libvirt-buildfail REGR. vs. 121247

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-xtf-amd64-amd64-2  50 xtf/test-hvm64-lbr-tsx-vmentry fail like 121093
 test-xtf-amd64-amd64-5  50 xtf/test-hvm64-lbr-tsx-vmentry fail like 121093
 test-armhf-armhf-xl-rtds 16 guest-start/debian.repeatfail  like 121093
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 121247
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 121247
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop fail like 121247
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 121247
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 121247
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop fail like 121247
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stopfail like 121247
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 121247
 test-xtf-amd64-amd64-3   52 xtf/test-hvm64-memop-seg fail   never pass
 test-xtf-amd64-amd64-2   52 xtf/test-hvm64-memop-seg fail   never pass
 test-xtf-amd64-amd64-4   52 xtf/test-hvm64-memop-seg fail   never pass
 test-xtf-amd64-amd64-1   52 xtf/test-hvm64-memop-seg fail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-xtf-amd64-amd64-5   52 xtf/test-hvm64-memop-seg fail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass

version targeted for testing:
 xen  

[Xen-devel] [xen-unstable-smoke test] 121346: tolerable all pass - PUSHED

[osstest service owner]

flight 121346 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/121346/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  ebe29cbd338aba99a0e17ecbdc73a25545bd219a
baseline version:
 xen  f809b61a0af2a659aef27db8d2bb1bb415b8c5e3

Last test of basis   121344  2018-03-29 15:02:10 Z0 days
Testing same since   121346  2018-03-29 18:01:05 Z0 days1 attempts


People who touched revisions under test:
  Jan Beulich 
  Olaf Hering 

jobs:
 build-arm64-xsm  pass
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   f809b61a0a..ebe29cbd33  ebe29cbd338aba99a0e17ecbdc73a25545bd219a -> smoke

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] seabios regression in staging

[Olaf Hering]

On Thu, Mar 29, Doug Goldstein wrote:

> If you'd be willing to create a SLE11 Docker container, we can add that
> to the tree and start doing builds against it.

I do not know how to do that. Any pointers?

Olaf


signature.asc
Description: PGP signature
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] seabios regression in staging

[Doug Goldstein]

On 3/29/18 9:53 AM, Olaf Hering wrote:
> On Thu, Mar 29, Wei Liu wrote:
> 
>> Do you use a non-default seabios configuration? Osstest seems to be
>> happy with the update.
> 
> Not sure how I would create a non-default seabios or toolchain build.
> 
> osstest does not use SLE11, so it can not possibly spot such compile
> errors. It would certainly be cool if there would be a test with very
> old and very new toolchain.
> 
> Olaf

If you'd be willing to create a SLE11 Docker container, we can add that
to the tree and start doing builds against it.

-- 
Doug Goldstein



signature.asc
Description: OpenPGP digital signature
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Upping gcc requirement for x86

[Doug Goldstein]

On 3/29/18 12:05 PM, George Dunlap wrote:
> On Thu, Mar 29, 2018 at 4:45 PM, Wei Liu  wrote:
> 
> Long term I think we want to get away from building seabios ourselves
> altogether; but it's a bit late in the release cycle to work out that
> kind of change.
> 
> On the whole I'd probably go with #3 at this point.
> 
>  -George
> 

Violent agreement here. Every distro wants to see this so they can share
these blobs. Its what lead to the change to tracking tags in
3dd926a25d866364ce6d46c21f9ac05a82fa7ffb originally.

-- 
Doug Goldstein

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] ARM: new VGIC: evtchn: fix potential race in vcpu_mark_events_pending()

[Stefano Stabellini]

On Thu, 29 Mar 2018, Andre Przywara wrote:
> Stefano pointed out the following situation:
> --
> 1) vcpuA/cpuA is running, it has already handled the event, cleared
> evtchn_upcall_pending and EOIed the event_irq but hasn't trapped into
> Xen yet. It is still in guest mode.
> 
> 2) Xen on cpuB calls vcpu_mark_events_pending(vcpuA), then calls
> vgic_inject_irq. However, because irq->line_level is high, it is not
> injected.
> 
> 3) vcpuA has to wait until trapping into Xen, calling
> vcpu_update_evtchn_irq, and going back to guest mode before receiving
> the event. This is theoretically a very long time.
> --
> 
> Fix this by updating the state of our emulated IRQ line level inside
> vcpu_mark_events_pending(), before trying to inject the new interrupt.
> 
> Despite having two calls to vgic_inject_irq(), only one will actually do
> something:
> - If the emulated line level was already in sync with the actual flag,
>   the VGIC ignores the first call, due to vgic_validate_injection().
> - If the emulated line level was high, but the flag says it should have
>   been low, vgic_inject_irq() will just update the line_level state.
> - If the emulated line level was low, but the flags says it should have
>   been high, we will inject the interrupt. The second call is then a
>   NOP.
> 
> Signed-off-by: Andre Przywara 
> ---
> Hi,
> 
> this would ideally have been part of a former patch:
> "[PATCH v3 06/39] ARM: evtchn: Handle level triggered IRQs correctly",
> but this has been merged already, so this has to be a follow-up.
> Ideally this would be merged before the final patch that introduces the
> CONFIG_NEW_VGIC Kconfig symbol, so that the old code gets never compiled.
> 
> Thanks,
> Andre
> 
>  xen/arch/arm/domain.c | 11 +--
>  1 file changed, 9 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index 9688e62f78..11fa9002dc 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -947,10 +947,17 @@ void vcpu_mark_events_pending(struct vcpu *v)
>  int already_pending = test_and_set_bit(
>  0, (unsigned long *)_info(v, evtchn_upcall_pending));
>  
> -if ( already_pending )
> -return;
> +#ifdef CONFIG_NEW_VGIC
> +/* Update the state of the current interrupt line. */
> +vgic_inject_irq(v->domain, v, v->domain->arch.evtchn_irq, 
> already_pending);
>  
> +/* Make the level IRQ pending. That's a NOP if it was already. */
>  vgic_inject_irq(v->domain, v, v->domain->arch.evtchn_irq, true);
> +#else
> +/* Only signal the VGIC if it wasn't already pending. */
> +if ( !already_pending )
> +vgic_inject_irq(v->domain, v, v->domain->arch.evtchn_irq, true);
> +#endif
>  }

The issue with this is that it is potentially racy, against vcpuA
trapping into Xen and executing vcpu_update_evtchn_irq, that also reads
evtchn_upcall_pending, then calls vgic_inject_irq. The last
vgic_inject_irq executed could be the one passing level = false, losing
the notification.

I might have a better idea: what if we just vcpu_kick(v) and do nothing
else? There is no need to call vgic_inject_irq from here because the
other vcpu will take care of doing it for us after trapping into Xen
(vcpu_update_evtchn_irq). It also needs to trap into Xen anyway to be
able to receive the new event as soon as possible.

The code would become:

  if ( already_pending )
  return;

  vgic_inject_irq(v->domain, v, v->domain->arch.evtchn_irq, true);

#ifdef CONFIG_NEW_VGIC
  vcpu_kick(v);
#endif


Note that vgic_inject_irq already does a vcpu_kick but only after
passing the vgic_validate_injection check, which would fail in this case
because irq->line_level is not up-to-date.

What do you think?

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Upping gcc requirement for x86

[George Dunlap]

On Thu, Mar 29, 2018 at 6:20 PM, Wei Liu  wrote:
> On Thu, Mar 29, 2018 at 06:14:08PM +0100, George Dunlap wrote:
>> On Thu, Mar 29, 2018 at 6:10 PM, Wei Liu  wrote:
>> > On Thu, Mar 29, 2018 at 06:05:57PM +0100, George Dunlap wrote:
>> >> On Thu, Mar 29, 2018 at 4:45 PM, Wei Liu  wrote:
>> >> > Hi all
>> >> >
>> >> > Seabios has bumped their requirement to 4.6 (released 7 years ago). We
>> >> > either need to bump our too or have a separate entry for seabios.
>> >>
>> >> RHEL / CentOS 6 are still supported, and they come with GCC 4.4.
>> >
>> > Where is this listed in xen.git? Supported in what sense?
>>
>> Sorry, I realized this was ambiguous just a minute ago.  I meant,
>> they're not EOL yet -- the distributions are still "active" as it
>> were.  (As opposed to, say, RHEL / CentOS 5, which is EOL.)
>>
>> I think it makes sense for us as a project to try to support
>> distributions which are still being given active support.
>
> Right. I think that makes sense, but does it actually work like that in
> practice?
>
> Existing Xen packages aren't really going to get a newer Xen, so that's
> out of the picture.
>
> What we try to achieve here is to let the users of these old distro able
> to build newer Xen themselves. But suppose you want to build a new
> system anyway, why stick with an old distro? Why not use a newer distro
> release instead?

Who knows why user do what they do? :-)  I'm sure there will be people
out there who do it; if it's not a terribly large effort, I think we
should try.  It builds good will.

 -George

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Upping gcc requirement for x86

[Wei Liu]

On Thu, Mar 29, 2018 at 06:14:08PM +0100, George Dunlap wrote:
> On Thu, Mar 29, 2018 at 6:10 PM, Wei Liu  wrote:
> > On Thu, Mar 29, 2018 at 06:05:57PM +0100, George Dunlap wrote:
> >> On Thu, Mar 29, 2018 at 4:45 PM, Wei Liu  wrote:
> >> > Hi all
> >> >
> >> > Seabios has bumped their requirement to 4.6 (released 7 years ago). We
> >> > either need to bump our too or have a separate entry for seabios.
> >>
> >> RHEL / CentOS 6 are still supported, and they come with GCC 4.4.
> >
> > Where is this listed in xen.git? Supported in what sense?
> 
> Sorry, I realized this was ambiguous just a minute ago.  I meant,
> they're not EOL yet -- the distributions are still "active" as it
> were.  (As opposed to, say, RHEL / CentOS 5, which is EOL.)
> 
> I think it makes sense for us as a project to try to support
> distributions which are still being given active support.

Right. I think that makes sense, but does it actually work like that in
practice?

Existing Xen packages aren't really going to get a newer Xen, so that's
out of the picture.

What we try to achieve here is to let the users of these old distro able
to build newer Xen themselves. But suppose you want to build a new
system anyway, why stick with an old distro? Why not use a newer distro
release instead?

Wei.

> 
>  -George

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-unstable-smoke test] 121344: tolerable all pass - PUSHED

[osstest service owner]

flight 121344 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/121344/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  f809b61a0af2a659aef27db8d2bb1bb415b8c5e3
baseline version:
 xen  e5fe34fd23816601de17b0a428909c95acf01c93

Last test of basis   121334  2018-03-28 19:15:21 Z0 days
Testing same since   121344  2018-03-29 15:02:10 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Jan Beulich 

jobs:
 build-arm64-xsm  pass
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   e5fe34fd23..f809b61a0a  f809b61a0af2a659aef27db8d2bb1bb415b8c5e3 -> smoke

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Upping gcc requirement for x86

[George Dunlap]

On Thu, Mar 29, 2018 at 6:10 PM, Wei Liu  wrote:
> On Thu, Mar 29, 2018 at 06:05:57PM +0100, George Dunlap wrote:
>> On Thu, Mar 29, 2018 at 4:45 PM, Wei Liu  wrote:
>> > Hi all
>> >
>> > Seabios has bumped their requirement to 4.6 (released 7 years ago). We
>> > either need to bump our too or have a separate entry for seabios.
>>
>> RHEL / CentOS 6 are still supported, and they come with GCC 4.4.
>
> Where is this listed in xen.git? Supported in what sense?

Sorry, I realized this was ambiguous just a minute ago.  I meant,
they're not EOL yet -- the distributions are still "active" as it
were.  (As opposed to, say, RHEL / CentOS 5, which is EOL.)

I think it makes sense for us as a project to try to support
distributions which are still being given active support.

 -George

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Upping gcc requirement for x86

[Wei Liu]

On Thu, Mar 29, 2018 at 06:05:57PM +0100, George Dunlap wrote:
> On Thu, Mar 29, 2018 at 4:45 PM, Wei Liu  wrote:
> > Hi all
> >
> > Seabios has bumped their requirement to 4.6 (released 7 years ago). We
> > either need to bump our too or have a separate entry for seabios.
> 
> RHEL / CentOS 6 are still supported, and they come with GCC 4.4.

Where is this listed in xen.git? Supported in what sense?

> 
> Other potential options:
> 
> 1. Have configure disable seabios if the gcc version is too old
> 2. Have configure change to an older seabios release if gcc is too old
> 3. Downgrade the seabios tag / changeset to a version that builds with
> older gcc versions
> 
> Long term I think we want to get away from building seabios ourselves
> altogether; but it's a bit late in the release cycle to work out that
> kind of change.
> 
> On the whole I'd probably go with #3 at this point.
> 
>  -George

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-4.6-testing test] 121328: regressions - FAIL

[osstest service owner]

flight 121328 xen-4.6-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/121328/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-qemuu-nested-intel 17 debian-hvm-install/l1/l2 fail REGR. vs. 
119227

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 15 guest-saverestore.2 
fail in 121311 pass in 121328
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 14 guest-saverestore.2 fail 
in 121311 pass in 121328
 test-amd64-i386-rumprun-i386 17 rumprun-demo-xenstorels/xenstorels.repeat fail 
pass in 121311
 test-amd64-i386-xl-qemut-ws16-amd64 15 guest-saverestore.2 fail pass in 121311

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-rtds 12 guest-start fail in 121311 like 119227
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop   fail in 121311 like 119227
 test-armhf-armhf-xl-rtds 16 guest-start/debian.repeatfail  like 119187
 test-xtf-amd64-amd64-5  50 xtf/test-hvm64-lbr-tsx-vmentry fail like 119227
 test-xtf-amd64-amd64-4  50 xtf/test-hvm64-lbr-tsx-vmentry fail like 119227
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 119227
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail  like 119227
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 119227
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 119227
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 119227
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 119227
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 119227
 test-xtf-amd64-amd64-2   37 xtf/test-hvm32pae-memop-seg  fail   never pass
 test-xtf-amd64-amd64-5   37 xtf/test-hvm32pae-memop-seg  fail   never pass
 test-xtf-amd64-amd64-1   37 xtf/test-hvm32pae-memop-seg  fail   never pass
 test-xtf-amd64-amd64-2   52 xtf/test-hvm64-memop-seg fail   never pass
 test-xtf-amd64-amd64-5   52 xtf/test-hvm64-memop-seg fail   never pass
 test-xtf-amd64-amd64-1   52 xtf/test-hvm64-memop-seg fail   never pass
 test-xtf-amd64-amd64-3   37 xtf/test-hvm32pae-memop-seg  fail   never pass
 test-xtf-amd64-amd64-5   76 xtf/test-pv32pae-xsa-194 fail   never pass
 test-xtf-amd64-amd64-4   37 xtf/test-hvm32pae-memop-seg  fail   never pass
 test-xtf-amd64-amd64-2   76 xtf/test-pv32pae-xsa-194 fail   never pass
 test-xtf-amd64-amd64-1   76 xtf/test-pv32pae-xsa-194 fail   never pass
 test-xtf-amd64-amd64-3   52 xtf/test-hvm64-memop-seg fail   never pass
 test-xtf-amd64-amd64-4   52 xtf/test-hvm64-memop-seg fail   never pass
 test-xtf-amd64-amd64-3   76 xtf/test-pv32pae-xsa-194 fail   never pass
 test-xtf-amd64-amd64-4   76 xtf/test-pv32pae-xsa-194 fail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 

Re: [Xen-devel] Upping gcc requirement for x86

[George Dunlap]

On Thu, Mar 29, 2018 at 4:45 PM, Wei Liu  wrote:
> Hi all
>
> Seabios has bumped their requirement to 4.6 (released 7 years ago). We
> either need to bump our too or have a separate entry for seabios.

RHEL / CentOS 6 are still supported, and they come with GCC 4.4.

Other potential options:

1. Have configure disable seabios if the gcc version is too old
2. Have configure change to an older seabios release if gcc is too old
3. Downgrade the seabios tag / changeset to a version that builds with
older gcc versions

Long term I think we want to get away from building seabios ourselves
altogether; but it's a bit late in the release cycle to work out that
kind of change.

On the whole I'd probably go with #3 at this point.

 -George

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Make coverity results public

[Stefano Stabellini]

On Thu, 29 Mar 2018, Roger Pau Monné wrote:
> On Thu, Mar 29, 2018 at 12:30:25AM +, Julien Grall wrote:
> > (sorry for the formatting)
> > 
> > On Wed, 28 Mar 2018, 21:48 George Dunlap,  wrote:
> > 
> > > On 03/28/2018 02:33 PM, Roger Pau Monné wrote:
> > > > Hello,
> > > >
> > > > According to the contribution guidelines document [0] the coverity
> > > > database of issues is private, which makes it hard for new people to
> > > > see issues. IMO it makes no sense to keep the result private anymore:
> > > >
> > > >  - They have been audited for plenty of time by different people
> > > >that currently has access to the database.
> > > >  - Anyone can reproduce the same results by forking Xen on github and
> > > >sending a build to coverity for analysis AFAICT.
> > > >
> > > > On the plus side, having the database open would allow us the
> > > > following:
> > > >
> > > >  - Coverity reports could be sent to xen-devel, so anyone could pick
> > > >and fix new issues.
> > > >  - Newcomers could use coverity in order to find small size tasks to
> > > >work on.
> > >
> > > In general, +1 from me.  But Stefano, was there some special
> > > circumstance for the ARM Coverity runs?
> > >
> > 
> > We don't control what is tested on the ARM coverity. This was setup on a
> > testing branch by EPAM, so they are putting their patches and update
> > manually.
> > 
> > If we want to open that coverity then we need to track staging and have
> > automatic push.
> > 
> > Otherwise it will be near to impossible to know if the failure is because
> > of staging or their patches.
> 
> I don't know much about Coverity, but I guess the results from the ARM
> scan are separated from the x86 ones?
> 
> Can't we just open the x86 results?
> 
> Or in the worse case, can we just ignore the ARM results if they are
> not useful?

Yes, there are basically two separate coverity instances: the one we
have been talking about and that Andrew was about to open up, which is
x86 only, and the one ran by EPAM, which is ARM only.

Let's take a step at a time and open up the x86 coverity instance first.
Then, we'll figure out what to do about the ARM instance.___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] xl/libxl: add pvcalls support

[Ian Jackson]

Stefano Stabellini writes ("[PATCH] xl/libxl: add pvcalls support"):
> Add pvcalls support to libxl and xl. Create the appropriate pvcalls
> entries in xenstore.
...
> + ~/device/pvcalls/$DEVID/* []
> +
> +Paravirtualized POSIX function calls frontend. Described by
> +[docs/misc/pvcalls.markdown][PVCALLS]

It's not entirely clear what the semantics are if multiple pvcalls
devices are provided.  Which is the guest expected to use ?

Perhaps the doc should state some convention, if there is one.  I hope
there is such a convention ($DEVID usually 0 maybe?)

> +for (i = 0; i < d_config->num_pvcallss; i++)

The name `pvcallss' is clumsy.  But I'm not sure I have a much better
suggestion.

One idea might be to rename your whole thing `pvrpc' since it's a
general RPC scheme, more or less.  Except that I don't want to come in
now and say you should rename it.  And also most rpc systems have an
idl language and you have ad hoc binary structs.

Have you considered calling this `pvcallsifs' ?  Where `if' is
`intrerface' ?  Or something ?

Anyway, I would like you to think about this and answer my questions
but I don't think it's a blocker. 

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] Freeze date for 4.11 shifted by one week

[Juergen Gross]

Hi all,

as the original freeze date (March 30th, 2018) is a holiday in many
countries and some of the maintainers have been very busy with
security work during most of the development phase of Xen 4.11 I've
decided to shift the freeze date of Xen 4.11 by one week.

So the new freeze date will be April 6th 2018.

Patches not being committed until then will miss Xen 4.11.

This is a one-time decision not automatically applying to future
releases.


Juergen

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 1/1] xen-netback: process malformed sk_buff correctly to avoid BUG_ON()

[David Miller]

From: Dongli Zhang 
Date: Wed, 28 Mar 2018 07:42:16 +0800

> The "BUG_ON(!frag_iter)" in function xenvif_rx_next_chunk() is triggered if
> the received sk_buff is malformed, that is, when the sk_buff has pattern
> (skb->data_len && !skb_shinfo(skb)->nr_frags). Below is a sample call
> stack:

We should fix the parts of the kernel which build illegal malformed
SKBs rather than adding checks to every driver in the tree.

I'm not applying this, sorry.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v19 10/11] common: add a new mappable resource type: XENMEM_resource_grant_table

[Paul Durrant]

This patch allows grant table frames to be mapped using the
XENMEM_acquire_resource memory op.

NOTE: This patch expands the on-stack mfn_list array in acquire_resource()
  but it is still small enough to remain on-stack.

Signed-off-by: Paul Durrant 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Ian Jackson 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Tim Deegan 
Cc: Wei Liu 

v19:
 - Add test to prevent PVH/HVM tools domains mapping grant status frames
   this way as the mapping infrastructure in Xen does not yet implement the
   necessary reference counting to make this safe.
 - Make sure grant table version is set before any attempt to grow the table.

v18:
 - Non-trivial re-base of grant table code.
 - Dropped Jan's R-b because of the grant table changes.

v13:
 - Re-work the internals to avoid using the XENMAPIDX_grant_table_status
   hack.

v12:
 - Dropped limit checks as requested by Jan.

v10:
 - Addressed comments from Jan.

v8:
 - The functionality was originally incorporated into the earlier patch
   "x86/mm: add HYPERVISOR_memory_op to acquire guest resources".
---
 xen/common/grant_table.c  | 74 +++
 xen/common/memory.c   | 56 +++-
 xen/include/public/memory.h   |  9 --
 xen/include/xen/grant_table.h |  4 +++
 4 files changed, 127 insertions(+), 16 deletions(-)

diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index 18201912e4..eba420a0a7 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -3863,6 +3863,38 @@ int mem_sharing_gref_to_gfn(struct grant_table *gt, 
grant_ref_t ref,
 }
 #endif
 
+/* caller must hold read or write lock */
+static int gnttab_get_status_frame_mfn(struct domain *d,
+   unsigned long idx, mfn_t *mfn)
+{
+struct grant_table *gt = d->grant_table;
+
+if ( idx >= nr_status_frames(gt) )
+return -EINVAL;
+
+*mfn = _mfn(virt_to_mfn(gt->status[idx]));
+return 0;
+}
+
+/* caller must hold write lock */
+static int gnttab_get_shared_frame_mfn(struct domain *d,
+   unsigned long idx, mfn_t *mfn)
+{
+struct grant_table *gt = d->grant_table;
+
+if ( gt->gt_version == 0 )
+gt->gt_version = 1;
+
+if ( (idx >= nr_grant_frames(gt)) && (idx < gt->max_grant_frames) )
+gnttab_grow_table(d, idx + 1);
+
+if ( idx >= nr_grant_frames(gt) )
+return -EINVAL;
+
+*mfn = _mfn(virt_to_mfn(gt->shared_raw[idx]));
+return 0;
+}
+
 int gnttab_map_frame(struct domain *d, unsigned long idx, gfn_t gfn,
  mfn_t *mfn)
 {
@@ -3880,21 +3912,11 @@ int gnttab_map_frame(struct domain *d, unsigned long 
idx, gfn_t gfn,
 {
 idx &= ~XENMAPIDX_grant_table_status;
 status = true;
-if ( idx < nr_status_frames(gt) )
-*mfn = _mfn(virt_to_mfn(gt->status[idx]));
-else
-rc = -EINVAL;
-}
-else
-{
-if ( (idx >= nr_grant_frames(gt)) && (idx < gt->max_grant_frames) )
-gnttab_grow_table(d, idx + 1);
 
-if ( idx < nr_grant_frames(gt) )
-*mfn = _mfn(virt_to_mfn(gt->shared_raw[idx]));
-else
-rc = -EINVAL;
+rc = gnttab_get_status_frame_mfn(d, idx, mfn);
 }
+else
+rc = gnttab_get_shared_frame_mfn(d, idx, mfn);
 
 if ( !rc && paging_mode_translate(d) &&
  !gfn_eq(gnttab_get_frame_gfn(gt, status, idx), INVALID_GFN) )
@@ -3909,6 +3931,32 @@ int gnttab_map_frame(struct domain *d, unsigned long 
idx, gfn_t gfn,
 return rc;
 }
 
+int gnttab_get_shared_frame(struct domain *d, unsigned long idx,
+mfn_t *mfn)
+{
+struct grant_table *gt = d->grant_table;
+int rc;
+
+grant_write_lock(gt);
+rc = gnttab_get_shared_frame_mfn(d, idx, mfn);
+grant_write_unlock(gt);
+
+return rc;
+}
+
+int gnttab_get_status_frame(struct domain *d, unsigned long idx,
+mfn_t *mfn)
+{
+struct grant_table *gt = d->grant_table;
+int rc;
+
+grant_read_lock(gt);
+rc = gnttab_get_status_frame_mfn(d, idx, mfn);
+grant_read_unlock(gt);
+
+return rc;
+}
+
 static void gnttab_usage_print(struct domain *rd)
 {
 int first = 1;
diff --git a/xen/common/memory.c b/xen/common/memory.c
index 2091bb8c2f..6a4744bd5d 100644
--- a/xen/common/memory.c
+++ b/xen/common/memory.c
@@ -23,6 +23,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -967,6 +968,54 @@ static long xatp_permission_check(struct domain *d, 
unsigned int space)
 return xsm_add_to_physmap(XSM_TARGET, current->domain, d);
 }
 
+static int acquire_grant_table(struct domain *d, unsigned int id,
+  

Re: [Xen-devel] Upping gcc requirement for x86

[Jan Beulich]

>>> On 29.03.18 at 17:45,  wrote:
> Seabios has bumped their requirement to 4.6 (released 7 years ago). We
> either need to bump our too or have a separate entry for seabios.

Ideally we would then come to common grounds with what the ARM
folks demand. I don't think we should have minimal requirements
imposed on ourselves by foreign projects we rely upon. And then I'm
pretty sure upstream qemu (just to give an example) doesn't care at
all about older gcc, yet also doesn't say what their minimum
requirement is.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] seabios regression in staging

[Wei Liu]

On Thu, Mar 29, 2018 at 05:36:32PM +0200, Olaf Hering wrote:
> On Thu, Mar 29, Wei Liu wrote:
> 
> > I think this is a problem with seabios upstream. We should ask them to
> > fix it and do another release.
> 
> https://mail.coreboot.org/pipermail/seabios/2017-November/011932.html
> 
> gcc-4.6+ is now required.
> 

Yeah. I saw that.

We should either bump our requirement or have a separate entry for
seabios. I prefer the former.

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v19 03/11] x86/hvm/ioreq: use gfn_t in struct hvm_ioreq_page

[Paul Durrant]

This patch adjusts the ioreq server code to use type-safe gfn_t values
where possible. No functional change.

Signed-off-by: Paul Durrant 
Reviewed-by: Roger Pau Monné 
Reviewed-by: Wei Liu 
Acked-by: Jan Beulich 
---
Cc: Andrew Cooper 

v18:
 - Trivial re-base.
---
 xen/arch/x86/hvm/ioreq.c | 46 
 xen/include/asm-x86/hvm/domain.h |  2 +-
 2 files changed, 24 insertions(+), 24 deletions(-)

diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index f5494a73c6..2512823de7 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -217,7 +217,7 @@ bool handle_hvm_io_completion(struct vcpu *v)
 return true;
 }
 
-static unsigned long hvm_alloc_ioreq_gfn(struct hvm_ioreq_server *s)
+static gfn_t hvm_alloc_ioreq_gfn(struct hvm_ioreq_server *s)
 {
 struct domain *d = s->target;
 unsigned int i;
@@ -227,20 +227,19 @@ static unsigned long hvm_alloc_ioreq_gfn(struct 
hvm_ioreq_server *s)
 for ( i = 0; i < sizeof(d->arch.hvm_domain.ioreq_gfn.mask) * 8; i++ )
 {
 if ( test_and_clear_bit(i, >arch.hvm_domain.ioreq_gfn.mask) )
-return d->arch.hvm_domain.ioreq_gfn.base + i;
+return _gfn(d->arch.hvm_domain.ioreq_gfn.base + i);
 }
 
-return gfn_x(INVALID_GFN);
+return INVALID_GFN;
 }
 
-static void hvm_free_ioreq_gfn(struct hvm_ioreq_server *s,
-   unsigned long gfn)
+static void hvm_free_ioreq_gfn(struct hvm_ioreq_server *s, gfn_t gfn)
 {
 struct domain *d = s->target;
-unsigned int i = gfn - d->arch.hvm_domain.ioreq_gfn.base;
+unsigned int i = gfn_x(gfn) - d->arch.hvm_domain.ioreq_gfn.base;
 
 ASSERT(!IS_DEFAULT(s));
-ASSERT(gfn != gfn_x(INVALID_GFN));
+ASSERT(!gfn_eq(gfn, INVALID_GFN));
 
 set_bit(i, >arch.hvm_domain.ioreq_gfn.mask);
 }
@@ -249,7 +248,7 @@ static void hvm_unmap_ioreq_gfn(struct hvm_ioreq_server *s, 
bool buf)
 {
 struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
 
-if ( iorp->gfn == gfn_x(INVALID_GFN) )
+if ( gfn_eq(iorp->gfn, INVALID_GFN) )
 return;
 
 destroy_ring_for_helper(>va, iorp->page);
@@ -258,7 +257,7 @@ static void hvm_unmap_ioreq_gfn(struct hvm_ioreq_server *s, 
bool buf)
 if ( !IS_DEFAULT(s) )
 hvm_free_ioreq_gfn(s, iorp->gfn);
 
-iorp->gfn = gfn_x(INVALID_GFN);
+iorp->gfn = INVALID_GFN;
 }
 
 static int hvm_map_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
@@ -271,16 +270,17 @@ static int hvm_map_ioreq_gfn(struct hvm_ioreq_server *s, 
bool buf)
 return -EINVAL;
 
 if ( IS_DEFAULT(s) )
-iorp->gfn = buf ?
-d->arch.hvm_domain.params[HVM_PARAM_BUFIOREQ_PFN] :
-d->arch.hvm_domain.params[HVM_PARAM_IOREQ_PFN];
+iorp->gfn = _gfn(buf ?
+ d->arch.hvm_domain.params[HVM_PARAM_BUFIOREQ_PFN] :
+ d->arch.hvm_domain.params[HVM_PARAM_IOREQ_PFN]);
 else
 iorp->gfn = hvm_alloc_ioreq_gfn(s);
 
-if ( iorp->gfn == gfn_x(INVALID_GFN) )
+if ( gfn_eq(iorp->gfn, INVALID_GFN) )
 return -ENOMEM;
 
-rc = prepare_ring_for_helper(d, iorp->gfn, >page, >va);
+rc = prepare_ring_for_helper(d, gfn_x(iorp->gfn), >page,
+ >va);
 
 if ( rc )
 hvm_unmap_ioreq_gfn(s, buf);
@@ -316,10 +316,10 @@ static void hvm_remove_ioreq_gfn(struct hvm_ioreq_server 
*s, bool buf)
 struct domain *d = s->target;
 struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
 
-if ( IS_DEFAULT(s) || iorp->gfn == gfn_x(INVALID_GFN) )
+if ( IS_DEFAULT(s) || gfn_eq(iorp->gfn, INVALID_GFN) )
 return;
 
-if ( guest_physmap_remove_page(d, _gfn(iorp->gfn),
+if ( guest_physmap_remove_page(d, iorp->gfn,
_mfn(page_to_mfn(iorp->page)), 0) )
 domain_crash(d);
 clear_page(iorp->va);
@@ -331,15 +331,15 @@ static int hvm_add_ioreq_gfn(struct hvm_ioreq_server *s, 
bool buf)
 struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
 int rc;
 
-if ( IS_DEFAULT(s) || iorp->gfn == gfn_x(INVALID_GFN) )
+if ( IS_DEFAULT(s) || gfn_eq(iorp->gfn, INVALID_GFN) )
 return 0;
 
 clear_page(iorp->va);
 
-rc = guest_physmap_add_page(d, _gfn(iorp->gfn),
+rc = guest_physmap_add_page(d, iorp->gfn,
 _mfn(page_to_mfn(iorp->page)), 0);
 if ( rc == 0 )
-paging_mark_pfn_dirty(d, _pfn(iorp->gfn));
+paging_mark_pfn_dirty(d, _pfn(gfn_x(iorp->gfn)));
 
 return rc;
 }
@@ -601,8 +601,8 @@ static int hvm_ioreq_server_init(struct hvm_ioreq_server *s,
 INIT_LIST_HEAD(>ioreq_vcpu_list);
 spin_lock_init(>bufioreq_lock);
 
-s->ioreq.gfn = gfn_x(INVALID_GFN);
-s->bufioreq.gfn = gfn_x(INVALID_GFN);
+s->ioreq.gfn = INVALID_GFN;
+s->bufioreq.gfn = INVALID_GFN;
 
 rc 

[Xen-devel] [PATCH v19 08/11] tools/libxenforeignmemory: add support for resource mapping

[Paul Durrant]

A previous patch introduced a new HYPERVISOR_memory_op to acquire guest
resources for direct priv-mapping.

This patch adds new functionality into libxenforeignmemory to make use
of a new privcmd ioctl [1] that uses the new memory op to make such
resources available via mmap(2).

[1] 
http://xenbits.xen.org/gitweb/?p=people/pauldu/linux.git;a=commit;h=ce59a05e6712

Signed-off-by: Paul Durrant 
Reviewed-by: Roger Pau Monné 
Reviewed-by: Wei Liu 
---
Cc: Ian Jackson 

v4:
 - Fixed errno and removed single-use label
 - The unmap call now returns a status
 - Use C99 initialization for ioctl struct

v2:
 - Bump minor version up to 3.
---
 tools/include/xen-sys/Linux/privcmd.h  | 11 +
 tools/libs/foreignmemory/Makefile  |  2 +-
 tools/libs/foreignmemory/core.c| 53 ++
 .../libs/foreignmemory/include/xenforeignmemory.h  | 41 +
 tools/libs/foreignmemory/libxenforeignmemory.map   |  5 ++
 tools/libs/foreignmemory/linux.c   | 45 ++
 tools/libs/foreignmemory/private.h | 31 +
 7 files changed, 187 insertions(+), 1 deletion(-)

diff --git a/tools/include/xen-sys/Linux/privcmd.h 
b/tools/include/xen-sys/Linux/privcmd.h
index 732ff7c15a..9531b728f9 100644
--- a/tools/include/xen-sys/Linux/privcmd.h
+++ b/tools/include/xen-sys/Linux/privcmd.h
@@ -86,6 +86,15 @@ typedef struct privcmd_dm_op {
const privcmd_dm_op_buf_t __user *ubufs;
 } privcmd_dm_op_t;
 
+typedef struct privcmd_mmap_resource {
+   domid_t dom;
+   __u32 type;
+   __u32 id;
+   __u32 idx;
+   __u64 num;
+   __u64 addr;
+} privcmd_mmap_resource_t;
+
 /*
  * @cmd: IOCTL_PRIVCMD_HYPERCALL
  * @arg: _hypercall_t
@@ -103,5 +112,7 @@ typedef struct privcmd_dm_op {
_IOC(_IOC_NONE, 'P', 5, sizeof(privcmd_dm_op_t))
 #define IOCTL_PRIVCMD_RESTRICT \
_IOC(_IOC_NONE, 'P', 6, sizeof(domid_t))
+#define IOCTL_PRIVCMD_MMAP_RESOURCE\
+   _IOC(_IOC_NONE, 'P', 7, sizeof(privcmd_mmap_resource_t))
 
 #endif /* __LINUX_PUBLIC_PRIVCMD_H__ */
diff --git a/tools/libs/foreignmemory/Makefile 
b/tools/libs/foreignmemory/Makefile
index cbe815fce8..ee5c3fd67e 100644
--- a/tools/libs/foreignmemory/Makefile
+++ b/tools/libs/foreignmemory/Makefile
@@ -2,7 +2,7 @@ XEN_ROOT = $(CURDIR)/../../..
 include $(XEN_ROOT)/tools/Rules.mk
 
 MAJOR= 1
-MINOR= 2
+MINOR= 3
 SHLIB_LDFLAGS += -Wl,--version-script=libxenforeignmemory.map
 
 CFLAGS   += -Werror -Wmissing-prototypes
diff --git a/tools/libs/foreignmemory/core.c b/tools/libs/foreignmemory/core.c
index 7c8562ae74..63f12e2450 100644
--- a/tools/libs/foreignmemory/core.c
+++ b/tools/libs/foreignmemory/core.c
@@ -17,6 +17,8 @@
 #include 
 #include 
 
+#include 
+
 #include "private.h"
 
 static int all_restrict_cb(Xentoolcore__Active_Handle *ah, domid_t domid) {
@@ -135,6 +137,57 @@ int xenforeignmemory_restrict(xenforeignmemory_handle 
*fmem,
 return osdep_xenforeignmemory_restrict(fmem, domid);
 }
 
+xenforeignmemory_resource_handle *xenforeignmemory_map_resource(
+xenforeignmemory_handle *fmem, domid_t domid, unsigned int type,
+unsigned int id, unsigned long frame, unsigned long nr_frames,
+void **paddr, int prot, int flags)
+{
+xenforeignmemory_resource_handle *fres;
+int rc;
+
+/* Check flags only contains POSIX defined values */
+if ( flags & ~(MAP_SHARED | MAP_PRIVATE) )
+{
+errno = EINVAL;
+return NULL;
+}
+
+fres = calloc(1, sizeof(*fres));
+if ( !fres )
+{
+errno = ENOMEM;
+return NULL;
+}
+
+fres->domid = domid;
+fres->type = type;
+fres->id = id;
+fres->frame = frame;
+fres->nr_frames = nr_frames;
+fres->addr = *paddr;
+fres->prot = prot;
+fres->flags = flags;
+
+rc = osdep_xenforeignmemory_map_resource(fmem, fres);
+if ( rc )
+{
+free(fres);
+fres = NULL;
+} else
+*paddr = fres->addr;
+
+return fres;
+}
+
+int xenforeignmemory_unmap_resource(
+xenforeignmemory_handle *fmem, xenforeignmemory_resource_handle *fres)
+{
+int rc = osdep_xenforeignmemory_unmap_resource(fmem, fres);
+
+free(fres);
+return rc;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libs/foreignmemory/include/xenforeignmemory.h 
b/tools/libs/foreignmemory/include/xenforeignmemory.h
index f4814c390f..d594be8df0 100644
--- a/tools/libs/foreignmemory/include/xenforeignmemory.h
+++ b/tools/libs/foreignmemory/include/xenforeignmemory.h
@@ -138,6 +138,47 @@ int xenforeignmemory_unmap(xenforeignmemory_handle *fmem,
 int xenforeignmemory_restrict(xenforeignmemory_handle *fmem,
   domid_t domid);
 
+typedef struct xenforeignmemory_resource_handle 
xenforeignmemory_resource_handle;
+
+/**
+ * This 

[Xen-devel] [PATCH v19 01/11] x86/hvm/ioreq: maintain an array of ioreq servers rather than a list

[Paul Durrant]

A subsequent patch will remove the current implicit limitation on creation
of ioreq servers which is due to the allocation of gfns for the ioreq
structures and buffered ioreq ring.

It will therefore be necessary to introduce an explicit limit and, since
this limit should be small, it simplifies the code to maintain an array of
that size rather than using a list.

Also, by reserving an array slot for the default server and populating
array slots early in create, the need to pass an 'is_default' boolean
to sub-functions can be avoided.

Some function return values are changed by this patch: Specifically, in
the case where the id of the default ioreq server is passed in, -EOPNOTSUPP
is now returned rather than -ENOENT.

Signed-off-by: Paul Durrant 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 

v19:
 - Improve comments.
 - Add missing checks for whether an ioreq server is enabled before
   sending emulation requests.

v18:
 - non-trivial re-base.
 - small modification to FOR_EACH... macro to iterate backwards, to main-
   tain a previous undocumented but useful semantic that secondary
   emulators are selected in favour of qemu.
 - dropped R-b's because of change.

v10:
 - modified FOR_EACH... macro as suggested by Jan.
 - check for NULL in IS_DEFAULT macro as suggested by Jan.

v9:
 - modified FOR_EACH... macro as requested by Andrew.

v8:
 - Addressed various comments from Jan.

v7:
 - Fixed assertion failure found in testing.

v6:
 - Updated according to comments made by Roger on v4 that I'd missed.

v5:
 - Switched GET/SET_IOREQ_SERVER() macros to get/set_ioreq_server()
   functions to avoid possible double-evaluation issues.

v4:
 - Introduced more helper macros and relocated them to the top of the
   code.

v3:
 - New patch (replacing "move is_default into struct hvm_ioreq_server") in
   response to review comments.
---
 xen/arch/x86/hvm/ioreq.c | 562 ---
 xen/include/asm-x86/hvm/domain.h |  11 +-
 2 files changed, 291 insertions(+), 282 deletions(-)

diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index 44d029499d..0a49fd7eb6 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -33,6 +33,44 @@
 
 #include 
 
+static void set_ioreq_server(struct domain *d, unsigned int id,
+ struct hvm_ioreq_server *s)
+{
+ASSERT(id < MAX_NR_IOREQ_SERVERS);
+ASSERT(!s || !d->arch.hvm_domain.ioreq_server.server[id]);
+
+d->arch.hvm_domain.ioreq_server.server[id] = s;
+}
+
+#define GET_IOREQ_SERVER(d, id) \
+(d)->arch.hvm_domain.ioreq_server.server[id]
+
+static struct hvm_ioreq_server *get_ioreq_server(const struct domain *d,
+ unsigned int id)
+{
+if ( id >= MAX_NR_IOREQ_SERVERS )
+return NULL;
+
+return GET_IOREQ_SERVER(d, id);
+}
+
+#define IS_DEFAULT(s) \
+((s) && (s) == GET_IOREQ_SERVER((s)->target, DEFAULT_IOSERVID))
+
+/*
+ * Iterate over all possible ioreq servers.
+ *
+ * NOTE: The iteration is backwards such that more recently created
+ *   ioreq servers are favoured in hvm_select_ioreq_server().
+ *   This is a semantic that previously existed when ioreq servers
+ *   were held in a linked list.
+ */
+#define FOR_EACH_IOREQ_SERVER(d, id, s) \
+for ( (id) = MAX_NR_IOREQ_SERVERS; (id) != 0; ) \
+if ( !(s = GET_IOREQ_SERVER(d, --(id))) ) \
+continue; \
+else
+
 static ioreq_t *get_ioreq(struct hvm_ioreq_server *s, struct vcpu *v)
 {
 shared_iopage_t *p = s->ioreq.va;
@@ -47,10 +85,9 @@ bool hvm_io_pending(struct vcpu *v)
 {
 struct domain *d = v->domain;
 struct hvm_ioreq_server *s;
+unsigned int id;
 
-list_for_each_entry ( s,
-  >arch.hvm_domain.ioreq_server.list,
-  list_entry )
+FOR_EACH_IOREQ_SERVER(d, id, s)
 {
 struct hvm_ioreq_vcpu *sv;
 
@@ -127,10 +164,9 @@ bool handle_hvm_io_completion(struct vcpu *v)
 struct hvm_vcpu_io *vio = >arch.hvm_vcpu.hvm_io;
 struct hvm_ioreq_server *s;
 enum hvm_io_completion io_completion;
+unsigned int id;
 
-  list_for_each_entry ( s,
-  >arch.hvm_domain.ioreq_server.list,
-  list_entry )
+FOR_EACH_IOREQ_SERVER(d, id, s)
 {
 struct hvm_ioreq_vcpu *sv;
 
@@ -243,13 +279,12 @@ static int hvm_map_ioreq_page(
 bool is_ioreq_server_page(struct domain *d, const struct page_info *page)
 {
 const struct hvm_ioreq_server *s;
+unsigned int id;
 bool found = false;
 
 spin_lock_recursive(>arch.hvm_domain.ioreq_server.lock);
 
-list_for_each_entry ( s,
-  >arch.hvm_domain.ioreq_server.list,
-  list_entry )
+FOR_EACH_IOREQ_SERVER(d, id, s)
 {
 if ( (s->ioreq.va && s->ioreq.page == page) ||
  (s->bufioreq.va && s->bufioreq.page == 

[Xen-devel] [PATCH v19 04/11] x86/hvm/ioreq: defer mapping gfns until they are actually requested

[Paul Durrant]

A subsequent patch will introduce a new scheme to allow an emulator to
map ioreq server pages directly from Xen rather than the guest P2M.

This patch lays the groundwork for that change by deferring mapping of
gfns until their values are requested by an emulator. To that end, the
pad field of the xen_dm_op_get_ioreq_server_info structure is re-purposed
to a flags field and new flag, XEN_DMOP_no_gfns, defined which modifies the
behaviour of XEN_DMOP_get_ioreq_server_info to allow the caller to avoid
requesting the gfn values.

Signed-off-by: Paul Durrant 
Reviewed-by: Roger Pau Monné 
Acked-by: Wei Liu 
Reviewed-by: Jan Beulich 
---
Cc: Ian Jackson 
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Tim Deegan 
Cc: Chao Gao 

v18:
 - Trivial re-base.

v17:
 - Fix typo in commit comment.

v16:
 - Leave call to map pages in hvm_ioreq_server_init() for default ioreq
   server instance, as pointed out by Chao (cc-ed). This is small and
   obvious change which reduces the size of the patch, so I have left
   existent R-bs and A-bs in place.

v8:
 - For safety make all of the pointers passed to
   hvm_get_ioreq_server_info() optional.
 - Shrink bufioreq_handling down to a uint8_t.

v3:
 - Updated in response to review comments from Wei and Roger.
 - Added a HANDLE_BUFIOREQ macro to make the code neater.
 - This patch no longer introduces a security vulnerability since there
   is now an explicit limit on the number of ioreq servers that may be
   created for any one domain.
---
 tools/libs/devicemodel/core.c   |  8 
 tools/libs/devicemodel/include/xendevicemodel.h |  6 +--
 xen/arch/x86/hvm/dm.c   |  9 +++--
 xen/arch/x86/hvm/ioreq.c| 49 -
 xen/include/asm-x86/hvm/domain.h|  2 +-
 xen/include/public/hvm/dm_op.h  | 32 +---
 6 files changed, 69 insertions(+), 37 deletions(-)

diff --git a/tools/libs/devicemodel/core.c b/tools/libs/devicemodel/core.c
index 23924e9a38..f76e3d305e 100644
--- a/tools/libs/devicemodel/core.c
+++ b/tools/libs/devicemodel/core.c
@@ -204,6 +204,14 @@ int xendevicemodel_get_ioreq_server_info(
 
 data->id = id;
 
+/*
+ * If the caller is not requesting gfn values then instruct the
+ * hypercall not to retrieve them as this may cause them to be
+ * mapped.
+ */
+if (!ioreq_gfn && !bufioreq_gfn)
+data->flags |= XEN_DMOP_no_gfns;
+
 rc = xendevicemodel_op(dmod, domid, 1, , sizeof(op));
 if (rc)
 return rc;
diff --git a/tools/libs/devicemodel/include/xendevicemodel.h 
b/tools/libs/devicemodel/include/xendevicemodel.h
index 7629c35df7..08cb0d4374 100644
--- a/tools/libs/devicemodel/include/xendevicemodel.h
+++ b/tools/libs/devicemodel/include/xendevicemodel.h
@@ -61,11 +61,11 @@ int xendevicemodel_create_ioreq_server(
  * @parm domid the domain id to be serviced
  * @parm id the IOREQ Server id.
  * @parm ioreq_gfn pointer to a xen_pfn_t to receive the synchronous ioreq
- *  gfn
+ *  gfn. (May be NULL if not required)
  * @parm bufioreq_gfn pointer to a xen_pfn_t to receive the buffered ioreq
- *gfn
+ *gfn. (May be NULL if not required)
  * @parm bufioreq_port pointer to a evtchn_port_t to receive the buffered
- * ioreq event channel
+ * ioreq event channel. (May be NULL if not required)
  * @return 0 on success, -1 on failure.
  */
 int xendevicemodel_get_ioreq_server_info(
diff --git a/xen/arch/x86/hvm/dm.c b/xen/arch/x86/hvm/dm.c
index 96b0d13f2f..ce18754442 100644
--- a/xen/arch/x86/hvm/dm.c
+++ b/xen/arch/x86/hvm/dm.c
@@ -420,16 +420,19 @@ static int dm_op(const struct dmop_args *op_args)
 {
 struct xen_dm_op_get_ioreq_server_info *data =
 _ioreq_server_info;
+const uint16_t valid_flags = XEN_DMOP_no_gfns;
 
 const_op = false;
 
 rc = -EINVAL;
-if ( data->pad )
+if ( data->flags & ~valid_flags )
 break;
 
 rc = hvm_get_ioreq_server_info(d, data->id,
-   >ioreq_gfn,
-   >bufioreq_gfn,
+   (data->flags & XEN_DMOP_no_gfns) ?
+   NULL : >ioreq_gfn,
+   (data->flags & XEN_DMOP_no_gfns) ?
+   NULL : >bufioreq_gfn,
>bufioreq_port);
 break;
 }
diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index 2512823de7..bb74f7881f 100644
--- 

[Xen-devel] [PATCH v19 02/11] x86/hvm/ioreq: simplify code and use consistent naming

[Paul Durrant]

This patch re-works much of the ioreq server initialization and teardown
code:

- The hvm_map/unmap_ioreq_gfn() functions are expanded to call through
  to hvm_alloc/free_ioreq_gfn() rather than expecting them to be called
  separately by outer functions.
- Several functions now test the validity of the hvm_ioreq_page gfn value
  to determine whether they need to act. This means can be safely called
  for the bufioreq page even when it is not used.
- hvm_add/remove_ioreq_gfn() simply return in the case of the default
  IOREQ server so callers no longer need to test before calling.
- hvm_ioreq_server_setup_pages() is renamed to hvm_ioreq_server_map_pages()
  to mirror the existing hvm_ioreq_server_unmap_pages().

All of this significantly shortens the code.

Signed-off-by: Paul Durrant 
Reviewed-by: Roger Pau Monné 
Reviewed-by: Wei Liu 
Acked-by: Jan Beulich 
---
Cc: Andrew Cooper 

v18:
 - Trivial re-base.

v3:
 - Re-based on top of 's->is_default' to 'IS_DEFAULT(s)' changes.
 - Minor updates in response to review comments from Roger.
---
 xen/arch/x86/hvm/ioreq.c | 182 ++-
 1 file changed, 69 insertions(+), 113 deletions(-)

diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index 0a49fd7eb6..f5494a73c6 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -217,63 +217,75 @@ bool handle_hvm_io_completion(struct vcpu *v)
 return true;
 }
 
-static int hvm_alloc_ioreq_gfn(struct domain *d, unsigned long *gfn)
+static unsigned long hvm_alloc_ioreq_gfn(struct hvm_ioreq_server *s)
 {
+struct domain *d = s->target;
 unsigned int i;
-int rc;
 
-rc = -ENOMEM;
+ASSERT(!IS_DEFAULT(s));
+
 for ( i = 0; i < sizeof(d->arch.hvm_domain.ioreq_gfn.mask) * 8; i++ )
 {
 if ( test_and_clear_bit(i, >arch.hvm_domain.ioreq_gfn.mask) )
-{
-*gfn = d->arch.hvm_domain.ioreq_gfn.base + i;
-rc = 0;
-break;
-}
+return d->arch.hvm_domain.ioreq_gfn.base + i;
 }
 
-return rc;
+return gfn_x(INVALID_GFN);
 }
 
-static void hvm_free_ioreq_gfn(struct domain *d, unsigned long gfn)
+static void hvm_free_ioreq_gfn(struct hvm_ioreq_server *s,
+   unsigned long gfn)
 {
+struct domain *d = s->target;
 unsigned int i = gfn - d->arch.hvm_domain.ioreq_gfn.base;
 
-if ( gfn != gfn_x(INVALID_GFN) )
-set_bit(i, >arch.hvm_domain.ioreq_gfn.mask);
+ASSERT(!IS_DEFAULT(s));
+ASSERT(gfn != gfn_x(INVALID_GFN));
+
+set_bit(i, >arch.hvm_domain.ioreq_gfn.mask);
 }
 
-static void hvm_unmap_ioreq_page(struct hvm_ioreq_server *s, bool buf)
+static void hvm_unmap_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
 {
 struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
 
+if ( iorp->gfn == gfn_x(INVALID_GFN) )
+return;
+
 destroy_ring_for_helper(>va, iorp->page);
+iorp->page = NULL;
+
+if ( !IS_DEFAULT(s) )
+hvm_free_ioreq_gfn(s, iorp->gfn);
+
+iorp->gfn = gfn_x(INVALID_GFN);
 }
 
-static int hvm_map_ioreq_page(
-struct hvm_ioreq_server *s, bool buf, unsigned long gfn)
+static int hvm_map_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
 {
 struct domain *d = s->target;
 struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
-struct page_info *page;
-void *va;
 int rc;
 
-if ( (rc = prepare_ring_for_helper(d, gfn, , )) )
-return rc;
-
-if ( (iorp->va != NULL) || d->is_dying )
-{
-destroy_ring_for_helper(, page);
+if ( d->is_dying )
 return -EINVAL;
-}
 
-iorp->va = va;
-iorp->page = page;
-iorp->gfn = gfn;
+if ( IS_DEFAULT(s) )
+iorp->gfn = buf ?
+d->arch.hvm_domain.params[HVM_PARAM_BUFIOREQ_PFN] :
+d->arch.hvm_domain.params[HVM_PARAM_IOREQ_PFN];
+else
+iorp->gfn = hvm_alloc_ioreq_gfn(s);
 
-return 0;
+if ( iorp->gfn == gfn_x(INVALID_GFN) )
+return -ENOMEM;
+
+rc = prepare_ring_for_helper(d, iorp->gfn, >page, >va);
+
+if ( rc )
+hvm_unmap_ioreq_gfn(s, buf);
+
+return rc;
 }
 
 bool is_ioreq_server_page(struct domain *d, const struct page_info *page)
@@ -286,8 +298,7 @@ bool is_ioreq_server_page(struct domain *d, const struct 
page_info *page)
 
 FOR_EACH_IOREQ_SERVER(d, id, s)
 {
-if ( (s->ioreq.va && s->ioreq.page == page) ||
- (s->bufioreq.va && s->bufioreq.page == page) )
+if ( (s->ioreq.page == page) || (s->bufioreq.page == page) )
 {
 found = true;
 break;
@@ -299,20 +310,30 @@ bool is_ioreq_server_page(struct domain *d, const struct 
page_info *page)
 return found;
 }
 
-static void hvm_remove_ioreq_gfn(
-struct domain *d, struct hvm_ioreq_page *iorp)
+static void hvm_remove_ioreq_gfn(struct hvm_ioreq_server 

[Xen-devel] [PATCH v19 05/11] x86/mm: add HYPERVISOR_memory_op to acquire guest resources

[Paul Durrant]

Certain memory resources associated with a guest are not necessarily
present in the guest P2M.

This patch adds the boilerplate for new memory op to allow such a resource
to be priv-mapped directly, by either a PV or HVM tools domain.

NOTE: Whilst the new op is not intrinsicly specific to the x86 architecture,
  I have no means to test it on an ARM platform and so cannot verify
  that it functions correctly.

Signed-off-by: Paul Durrant 
Acked-by: Daniel De Graaf 
Reviewed-by: Jan Beulich 
---
Cc: George Dunlap 
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Ian Jackson 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Tim Deegan 
Cc: Wei Liu 
Cc: Julien Grall 

v19:
 - Small error path tweak suggested by Jan.
 - Flag name change requested by Jan.

v18:
 - Allow the resource page owner to be specified by a returned flag.
 - Drop Jan's R-b due to change.

v14:
 - Addressed more comments from Jan.

v13:
 - Use xen_pfn_t for mfn_list.
 - Addressed further comments from Jan and Julien.

v12:
 - Addressed more comments form Jan.
 - Removed #ifdef CONFIG_X86 from common code and instead introduced a
   stub set_foreign_p2m_entry() in asm-arm/p2m.h returning -EOPNOTSUPP.
 - Restricted mechanism for querying implementation limit on nr_frames
   and simplified compat code.

v11:
 - Addressed more comments from Jan.

v9:
 - Addressed more comments from Jan.

v8:
 - Move the code into common as requested by Jan.
 - Make the gmfn_list handle a 64-bit type to avoid limiting the MFN
   range for a 32-bit tools domain.
 - Add missing pad.
 - Add compat code.
 - Make this patch deal with purely boilerplate.
 - Drop George's A-b and Wei's R-b because the changes are non-trivial,
   and update Cc list now the boilerplate is common.

v5:
 - Switched __copy_to/from_guest_offset() to copy_to/from_guest_offset().
---
 tools/flask/policy/modules/xen.if   |   4 +-
 xen/arch/x86/mm/p2m.c   |   3 +-
 xen/common/compat/memory.c  | 100 
 xen/common/memory.c |  91 
 xen/include/asm-arm/p2m.h   |  10 
 xen/include/asm-x86/p2m.h   |   3 ++
 xen/include/public/memory.h |  55 +++-
 xen/include/xlat.lst|   1 +
 xen/include/xsm/dummy.h |   6 +++
 xen/include/xsm/xsm.h   |   6 +++
 xen/xsm/dummy.c |   1 +
 xen/xsm/flask/hooks.c   |   6 +++
 xen/xsm/flask/policy/access_vectors |   2 +
 13 files changed, 284 insertions(+), 4 deletions(-)

diff --git a/tools/flask/policy/modules/xen.if 
b/tools/flask/policy/modules/xen.if
index 459880bb01..7aefd0061e 100644
--- a/tools/flask/policy/modules/xen.if
+++ b/tools/flask/policy/modules/xen.if
@@ -52,7 +52,8 @@ define(`create_domain_common', `
settime setdomainhandle getvcpucontext set_misc_info };
allow $1 $2:domain2 { set_cpuid settsc setscheduler setclaim
set_max_evtchn set_vnumainfo get_vnumainfo cacheflush
-   psr_cmt_op psr_alloc soft_reset set_gnttab_limits };
+   psr_cmt_op psr_alloc soft_reset set_gnttab_limits
+   resource_map };
allow $1 $2:security check_context;
allow $1 $2:shadow enable;
allow $1 $2:mmu { map_read map_write adjust memorymap physmap pinpage 
mmuext_op updatemp };
@@ -152,6 +153,7 @@ define(`device_model', `
allow $1 $2_target:domain { getdomaininfo shutdown };
allow $1 $2_target:mmu { map_read map_write adjust physmap target_hack 
};
allow $1 $2_target:hvm { getparam setparam hvmctl dm };
+   allow $1 $2_target:domain2 resource_map;
 ')
 
 # make_device_model(priv, dm_dom, hvm_dom)
diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
index 48e50fb5d8..55693eba59 100644
--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -1132,8 +1132,7 @@ static int set_typed_p2m_entry(struct domain *d, unsigned 
long gfn_l,
 }
 
 /* Set foreign mfn in the given guest's p2m table. */
-static int set_foreign_p2m_entry(struct domain *d, unsigned long gfn,
- mfn_t mfn)
+int set_foreign_p2m_entry(struct domain *d, unsigned long gfn, mfn_t mfn)
 {
 return set_typed_p2m_entry(d, gfn, mfn, PAGE_ORDER_4K, p2m_map_foreign,
p2m_get_hostp2m(d)->default_access);
diff --git a/xen/common/compat/memory.c b/xen/common/compat/memory.c
index 35bb259808..13fd64ddf5 100644
--- a/xen/common/compat/memory.c
+++ b/xen/common/compat/memory.c
@@ -71,6 +71,7 @@ int compat_memory_op(unsigned int cmd, 
XEN_GUEST_HANDLE_PARAM(void) compat)
 struct 

[Xen-devel] [PATCH v19 07/11] x86/mm: add an extra command to HYPERVISOR_mmu_update...

[Paul Durrant]

...to allow the calling domain to prevent translation of specified l1e
value.

Despite what the comment in public/xen.h might imply, specifying a
command value of MMU_NORMAL_PT_UPDATE will not simply update an l1e with
the specified value. Instead, mod_l1_entry() tests whether foreign_dom
has PG_translate set in its paging mode and, if it does, assumes that the
the pfn value in the l1e is a gfn rather than an mfn.

To allow PV tools domain to map mfn values from a previously issued
HYPERVISOR_memory_op:XENMEM_acquire_resource, there needs to be a way
to tell HYPERVISOR_mmu_update that the specific l1e value does not
require translation regardless of the paging mode of foreign_dom. This
patch therefore defines a new command value, MMU_PT_UPDATE_NO_TRANSLATE,
which has the same semantics as MMU_NORMAL_PT_UPDATE except that the
paging mode of foreign_dom is ignored and the l1e value is used verbatim.

Signed-off-by: Paul Durrant 
Reviewed-by: Jan Beulich 
---
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Ian Jackson 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Tim Deegan 
Cc: Wei Liu 

v13:
 - Re-base.

v8:
 - New in this version, replacing "allow a privileged PV domain to map
   guest mfns".
---
 xen/arch/x86/mm.c| 13 -
 xen/include/public/xen.h | 12 +---
 2 files changed, 17 insertions(+), 8 deletions(-)

diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index 4964910d09..fcfaae19c9 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -1901,9 +1901,10 @@ void page_unlock(struct page_info *page)
 
 /* Update the L1 entry at pl1e to new value nl1e. */
 static int mod_l1_entry(l1_pgentry_t *pl1e, l1_pgentry_t nl1e,
-unsigned long gl1mfn, int preserve_ad,
+unsigned long gl1mfn, unsigned int cmd,
 struct vcpu *pt_vcpu, struct domain *pg_dom)
 {
+bool preserve_ad = (cmd == MMU_PT_UPDATE_PRESERVE_AD);
 l1_pgentry_t ol1e;
 struct domain *pt_dom = pt_vcpu->domain;
 int rc = 0;
@@ -1925,7 +1926,8 @@ static int mod_l1_entry(l1_pgentry_t *pl1e, l1_pgentry_t 
nl1e,
 }
 
 /* Translate foreign guest address. */
-if ( paging_mode_translate(pg_dom) )
+if ( cmd != MMU_PT_UPDATE_NO_TRANSLATE &&
+ paging_mode_translate(pg_dom) )
 {
 p2m_type_t p2mt;
 p2m_query_t q = l1e_get_flags(nl1e) & _PAGE_RW ?
@@ -3617,6 +3619,7 @@ long do_mmu_update(
  */
 case MMU_NORMAL_PT_UPDATE:
 case MMU_PT_UPDATE_PRESERVE_AD:
+case MMU_PT_UPDATE_NO_TRANSLATE:
 {
 p2m_type_t p2mt;
 
@@ -3676,8 +3679,7 @@ long do_mmu_update(
 {
 case PGT_l1_page_table:
 rc = mod_l1_entry(va, l1e_from_intpte(req.val), mfn,
-  cmd == MMU_PT_UPDATE_PRESERVE_AD, v,
-  pg_owner);
+  cmd, v, pg_owner);
 break;
 
 case PGT_l2_page_table:
@@ -3988,7 +3990,8 @@ static int __do_update_va_mapping(
 goto out;
 }
 
-rc = mod_l1_entry(pl1e, val, mfn_x(gl1mfn), 0, v, pg_owner);
+rc = mod_l1_entry(pl1e, val, mfn_x(gl1mfn), MMU_NORMAL_PT_UPDATE, v,
+  pg_owner);
 
 page_unlock(gl1pg);
 put_page(gl1pg);
diff --git a/xen/include/public/xen.h b/xen/include/public/xen.h
index 308109f176..fb1df8f293 100644
--- a/xen/include/public/xen.h
+++ b/xen/include/public/xen.h
@@ -268,6 +268,10 @@ DEFINE_XEN_GUEST_HANDLE(xen_ulong_t);
  * As MMU_NORMAL_PT_UPDATE above, but A/D bits currently in the PTE are ORed
  * with those in @val.
  *
+ * ptr[1:0] == MMU_PT_UPDATE_NO_TRANSLATE:
+ * As MMU_NORMAL_PT_UPDATE above, but @val is not translated though FD
+ * page tables.
+ *
  * @val is usually the machine frame number along with some attributes.
  * The attributes by default follow the architecture defined bits. Meaning that
  * if this is a X86_64 machine and four page table layout is used, the layout
@@ -334,9 +338,11 @@ DEFINE_XEN_GUEST_HANDLE(xen_ulong_t);
  *
  * PAT (bit 7 on) --> PWT (bit 3 on) and clear bit 7.
  */
-#define MMU_NORMAL_PT_UPDATE  0 /* checked '*ptr = val'. ptr is MA.  */
-#define MMU_MACHPHYS_UPDATE   1 /* ptr = MA of frame to modify entry for */
-#define MMU_PT_UPDATE_PRESERVE_AD 2 /* atomically: *ptr = val | (*ptr&(A|D)) */
+#define MMU_NORMAL_PT_UPDATE   0 /* checked '*ptr = val'. ptr is MA.  
*/
+#define MMU_MACHPHYS_UPDATE1 /* ptr = MA of frame to modify entry for 
*/
+#define MMU_PT_UPDATE_PRESERVE_AD  2 /* atomically: *ptr = val | (*ptr&(A|D)) 
*/
+#define MMU_PT_UPDATE_NO_TRANSLATE 3 /* checked '*ptr = val'. ptr is MA.  
*/
+  

[Xen-devel] [PATCH v19 06/11] x86/hvm/ioreq: add a new mappable resource type...

[Paul Durrant]

... XENMEM_resource_ioreq_server

This patch adds support for a new resource type that can be mapped using
the XENMEM_acquire_resource memory op.

If an emulator makes use of this resource type then, instead of mapping
gfns, the IOREQ server will allocate pages which are assigned to the
emulating domain. These pages will never be present in the P2M of the
guest at any point (and are not even shared with the guest) and so are not
vulnerable to any direct attack by the guest.

NOTE: Use of the new resource type is not compatible with use of
  XEN_DMOP_get_ioreq_server_info unless the XEN_DMOP_no_gfns flag is
  set.

Signed-off-by: Paul Durrant 
---
Cc: Jan Beulich 
Cc: George Dunlap 
Cc: Wei Liu 
Cc: Andrew Cooper 
Cc: Ian Jackson 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Tim Deegan 
Cc: Julien Grall 

v19:
 - Fix error path in hvm_alloc_ioreq_mfn().
 - Tweak comment and flag name in arch_acquire_resource().
 - Add missing blank in asm-arm/mm.h declaration.

v18:
 - Revert largely back to v14, but use ioreq server emulator rather
   than current->domain.
 - Add missing checks spotted by Jan.
 - Re-base.

v17:
 - The use of xenheap pages means that freeing needs to be deferred until
   domain destruction. Add an explanatory paragraph to the commit comment.

v15:
 - Use xenheap pages rather than domheap pages and assign ownership to
   target domain.

v14:
 - Addressed more comments from Jan.

v13:
 - Introduce an arch_acquire_resource() as suggested by Julien (and have
   the ARM varient simply return -EOPNOTSUPP).
 - Check for ioreq server id truncation as requested by Jan.
 - Not added Jan's R-b due to substantive change from v12.

v12:
 - Addressed more comments from Jan.
 - Dropped George's A-b and Wei's R-b because of material change.

v11:
 - Addressed more comments from Jan.

v10:
 - Addressed comments from Jan.

v8:
 - Re-base on new boilerplate.
 - Adjust function signature of hvm_get_ioreq_server_frame(), and test
   whether the bufioreq page is present.

v5:
 - Use get_ioreq_server() function rather than indexing array directly.
 - Add more explanation into comments to state than mapping guest frames
   and allocation of pages for ioreq servers are not simultaneously
   permitted.
 - Add a comment into asm/ioreq.h stating the meaning of the index
   value passed to hvm_get_ioreq_server_frame().
---
 xen/arch/x86/hvm/ioreq.c| 167 
 xen/arch/x86/mm.c   |  47 +++
 xen/common/memory.c |   3 +-
 xen/include/asm-arm/mm.h|   8 ++
 xen/include/asm-x86/hvm/ioreq.h |   2 +
 xen/include/asm-x86/mm.h|   5 ++
 xen/include/public/hvm/dm_op.h  |   4 +
 xen/include/public/memory.h |   9 +++
 8 files changed, 244 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index bb74f7881f..269293f773 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -266,6 +266,19 @@ static int hvm_map_ioreq_gfn(struct hvm_ioreq_server *s, 
bool buf)
 struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
 int rc;
 
+if ( iorp->page )
+{
+/*
+ * If a page has already been allocated (which will happen on
+ * demand if hvm_get_ioreq_server_frame() is called), then
+ * mapping a guest frame is not permitted.
+ */
+if ( gfn_eq(iorp->gfn, INVALID_GFN) )
+return -EPERM;
+
+return 0;
+}
+
 if ( d->is_dying )
 return -EINVAL;
 
@@ -288,6 +301,70 @@ static int hvm_map_ioreq_gfn(struct hvm_ioreq_server *s, 
bool buf)
 return rc;
 }
 
+static int hvm_alloc_ioreq_mfn(struct hvm_ioreq_server *s, bool buf)
+{
+struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
+
+if ( iorp->page )
+{
+/*
+ * If a guest frame has already been mapped (which may happen
+ * on demand if hvm_get_ioreq_server_info() is called), then
+ * allocating a page is not permitted.
+ */
+if ( !gfn_eq(iorp->gfn, INVALID_GFN) )
+return -EPERM;
+
+return 0;
+}
+
+/*
+ * Allocated IOREQ server pages are assigned to the emulating
+ * domain, not the target domain. This is safe because the emulating
+ * domain cannot be destroyed until the ioreq server is destroyed.
+ * Also we must use MEMF_no_refcount otherwise page allocation
+ * could fail if the emulating domain has already reached its
+ * maximum allocation.
+ */
+iorp->page = alloc_domheap_page(s->emulator, MEMF_no_refcount);
+
+if ( !iorp->page )
+return -ENOMEM;
+
+if ( !get_page_type(iorp->page, PGT_writable_page) )
+goto fail1;
+
+iorp->va = 

[Xen-devel] [PATCH v19 00/11] x86: guest resource mapping

[Paul Durrant]

This series introduces support for direct mapping of guest resources.
The resources are:
 - IOREQ server pages
 - Grant tables

v19:
 - Respond to Jan's latest comments and fix grant table verion setting lost in 
re-base

v18:
 - Re-base
 - Use the now-reference-counted emulating domain to host ioreq pages

v17:
 - Make sure ioreq page free-ing is done at domain destruction

v16:
 - Fix default ioreq server code and verified with qemu trad

v15:
 - Correct page ownership of ioreq pages

v14:
 - Responded to more comments from Jan.

v13:
 - Responded to more comments from Jan and Julien.
 - Build-tested using ARM cross-compilation.

v12:
 - Responded to more comments from Jan.

v11:
 - Responded to more comments from Jan.

v10:
 - Responded to comments from Jan.

v9:
 - Change to patch #1 only.

v8:
 - Re-ordered series and dropped two patches that have already been
   committed.

v7:
 - Fixed assertion failure hit during domain destroy.

v6:
 - Responded to missed comments from Roger.

v5:
 - Responded to review comments from Wei.

v4:
 - Responded to further review comments from Roger.

v3:
 - Dropped original patch #1 since it is covered by Juergen's patch.
 - Added new xenforeignmemorycleanup patch (#4).
 - Replaced the patch introducing the ioreq server 'is_default' flag with
   one that changes the ioreq server list into an array (#8).

Paul Durrant (11):
  x86/hvm/ioreq: maintain an array of ioreq servers rather than a list
  x86/hvm/ioreq: simplify code and use consistent naming
  x86/hvm/ioreq: use gfn_t in struct hvm_ioreq_page
  x86/hvm/ioreq: defer mapping gfns until they are actually requested
  x86/mm: add HYPERVISOR_memory_op to acquire guest resources
  x86/hvm/ioreq: add a new mappable resource type...
  x86/mm: add an extra command to HYPERVISOR_mmu_update...
  tools/libxenforeignmemory: add support for resource mapping
  tools/libxenforeignmemory: reduce xenforeignmemory_restrict code
footprint
  common: add a new mappable resource type: XENMEM_resource_grant_table
  tools/libxenctrl: use new xenforeignmemory API to seed grant table

 tools/flask/policy/modules/xen.if  |   4 +-
 tools/include/xen-sys/Linux/privcmd.h  |  11 +
 tools/libs/devicemodel/core.c  |   8 +
 tools/libs/devicemodel/include/xendevicemodel.h|   6 +-
 tools/libs/foreignmemory/Makefile  |   2 +-
 tools/libs/foreignmemory/core.c|  53 ++
 tools/libs/foreignmemory/freebsd.c |   7 -
 .../libs/foreignmemory/include/xenforeignmemory.h  |  41 +
 tools/libs/foreignmemory/libxenforeignmemory.map   |   5 +
 tools/libs/foreignmemory/linux.c   |  45 +
 tools/libs/foreignmemory/minios.c  |   7 -
 tools/libs/foreignmemory/netbsd.c  |   7 -
 tools/libs/foreignmemory/private.h |  43 +-
 tools/libs/foreignmemory/solaris.c |   7 -
 tools/libxc/include/xc_dom.h   |   8 +-
 tools/libxc/xc_dom_boot.c  | 114 ++-
 tools/libxc/xc_sr_restore_x86_hvm.c|  10 +-
 tools/libxc/xc_sr_restore_x86_pv.c |   2 +-
 tools/libxl/libxl_dom.c|   1 -
 tools/python/xen/lowlevel/xc/xc.c  |   6 +-
 xen/arch/x86/hvm/dm.c  |   9 +-
 xen/arch/x86/hvm/ioreq.c   | 910 -
 xen/arch/x86/mm.c  |  60 +-
 xen/arch/x86/mm/p2m.c  |   3 +-
 xen/common/compat/memory.c | 100 +++
 xen/common/grant_table.c   |  74 +-
 xen/common/memory.c| 146 
 xen/include/asm-arm/mm.h   |   8 +
 xen/include/asm-arm/p2m.h  |  10 +
 xen/include/asm-x86/hvm/domain.h   |  15 +-
 xen/include/asm-x86/hvm/ioreq.h|   2 +
 xen/include/asm-x86/mm.h   |   5 +
 xen/include/asm-x86/p2m.h  |   3 +
 xen/include/public/hvm/dm_op.h |  36 +-
 xen/include/public/memory.h|  69 +-
 xen/include/public/xen.h   |  12 +-
 xen/include/xen/grant_table.h  |   4 +
 xen/include/xlat.lst   |   1 +
 xen/include/xsm/dummy.h|   6 +
 xen/include/xsm/xsm.h  |   6 +
 xen/xsm/dummy.c|   1 +
 xen/xsm/flask/hooks.c  |   6 +
 xen/xsm/flask/policy/access_vectors|   2 +
 43 files changed, 1361 insertions(+), 514 deletions(-)
---
Cc: Daniel De Graaf 
Cc: Ian Jackson 
Cc: Wei Liu 
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: 

[Xen-devel] Upping gcc requirement for x86

[Wei Liu]

Hi all

Seabios has bumped their requirement to 4.6 (released 7 years ago). We
either need to bump our too or have a separate entry for seabios.

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v4 7/7] xen/x86: use PCID feature

[Jan Beulich]

>>> On 29.03.18 at 17:15,  wrote:
> On 29/03/18 16:19, Jan Beulich wrote:
> On 27.03.18 at 11:07,  wrote:
>>> @@ -102,7 +103,21 @@ void write_cr3_cr4(unsigned long cr3, unsigned long 
>>> cr4)
>>>  t = pre_flush();
>>>  
>>>  if ( read_cr4() & X86_CR4_PGE )
>>> +/*
>>> + * X86_CR4_PGE set means PCID being inactive.
>>> + * We have to purge the TLB via flipping cr4.pge.
>>> + */
>>>  write_cr4(cr4 & ~X86_CR4_PGE);
>>> +else if ( cpu_has_invpcid )
>>> +/*
>>> + * If we are using PCID purge the TLB via INVPCID as loading cr3
>>> + * will affect the current PCID only.
>> 
>> s/current/new/ ?
> 
> Okay.
> 
>> 
>>> + * If INVPCID is not supported we don't use PCIDs so loading cr3
>>> + * will purge the TLB (we are in the "global pages off" branch).
>>> + * invpcid_flush_all_nonglobals() seems to be faster than
>>> + * invpcid_flush_all().
>>> + */
>>> +invpcid_flush_all_nonglobals();
>>>  
>>>  asm volatile ( "mov %0, %%cr3" : : "r" (cr3) : "memory" );
>> 
>> What about the TLB entries that have been re-created between
>> the INVPCID and the write of CR3? It's not obvious to me that
>> leaving them around is not going to be a problem subsequently,
>> the more that you write cr3 frequently with the no-flush bit set.
> 
> The no-flush bit should not make any difference here. It controls only
> flushing of TLB-entries with the new PCID.

Right, but in a subsequent write to CR3 you may make active again
what was the old PCID here. This is in particular so for PCID 0 (which
has dual use).

>> Don't you need to do a single context invalidation for the prior
>> PCID (if different from the new one)?
> 
> Hmm, I don't think so, as the mov to %cr3 is a serializing instruction
> acting as a speculation barrier. So the only TLB entries which could
> survive would be the ones for the few instruction bytes after the
> invpcid instruction until the end of the mov to %cr3. Those are harmless
> as they are associated with the hypervisor PCID value, so there is no
> risk of any data leak to a guest. Maybe a comment explaining that would
> be a good idea.

Well, to be honest I don't trust in things like "speculation barrier"
anymore, at least not as far as things ahead of the barrier go.
Who knows what exactly the CPU does (and hence which TLB
entries it creates) between the INVPCID and the CR3 write. I
don't think we can blindly assume only entries for Xen mappings
could be created during that window.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] seabios regression in staging

[Olaf Hering]

On Thu, Mar 29, Wei Liu wrote:

> I think this is a problem with seabios upstream. We should ask them to
> fix it and do another release.

https://mail.coreboot.org/pipermail/seabios/2017-November/011932.html

gcc-4.6+ is now required.

Olaf


signature.asc
Description: PGP signature
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH] ARM: new VGIC: evtchn: fix potential race in vcpu_mark_events_pending()

[Andre Przywara]

Stefano pointed out the following situation:
--
1) vcpuA/cpuA is running, it has already handled the event, cleared
evtchn_upcall_pending and EOIed the event_irq but hasn't trapped into
Xen yet. It is still in guest mode.

2) Xen on cpuB calls vcpu_mark_events_pending(vcpuA), then calls
vgic_inject_irq. However, because irq->line_level is high, it is not
injected.

3) vcpuA has to wait until trapping into Xen, calling
vcpu_update_evtchn_irq, and going back to guest mode before receiving
the event. This is theoretically a very long time.
--

Fix this by updating the state of our emulated IRQ line level inside
vcpu_mark_events_pending(), before trying to inject the new interrupt.

Despite having two calls to vgic_inject_irq(), only one will actually do
something:
- If the emulated line level was already in sync with the actual flag,
  the VGIC ignores the first call, due to vgic_validate_injection().
- If the emulated line level was high, but the flag says it should have
  been low, vgic_inject_irq() will just update the line_level state.
- If the emulated line level was low, but the flags says it should have
  been high, we will inject the interrupt. The second call is then a
  NOP.

Signed-off-by: Andre Przywara 
---
Hi,

this would ideally have been part of a former patch:
"[PATCH v3 06/39] ARM: evtchn: Handle level triggered IRQs correctly",
but this has been merged already, so this has to be a follow-up.
Ideally this would be merged before the final patch that introduces the
CONFIG_NEW_VGIC Kconfig symbol, so that the old code gets never compiled.

Thanks,
Andre

 xen/arch/arm/domain.c | 11 +--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 9688e62f78..11fa9002dc 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -947,10 +947,17 @@ void vcpu_mark_events_pending(struct vcpu *v)
 int already_pending = test_and_set_bit(
 0, (unsigned long *)_info(v, evtchn_upcall_pending));
 
-if ( already_pending )
-return;
+#ifdef CONFIG_NEW_VGIC
+/* Update the state of the current interrupt line. */
+vgic_inject_irq(v->domain, v, v->domain->arch.evtchn_irq, already_pending);
 
+/* Make the level IRQ pending. That's a NOP if it was already. */
 vgic_inject_irq(v->domain, v, v->domain->arch.evtchn_irq, true);
+#else
+/* Only signal the VGIC if it wasn't already pending. */
+if ( !already_pending )
+vgic_inject_irq(v->domain, v, v->domain->arch.evtchn_irq, true);
+#endif
 }
 
 void vcpu_update_evtchn_irq(struct vcpu *v)
-- 
2.14.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v4 7/7] xen/x86: use PCID feature

[Juergen Gross]

On 29/03/18 16:19, Jan Beulich wrote:
 On 27.03.18 at 11:07,  wrote:
>> --- a/xen/arch/x86/domain_page.c
>> +++ b/xen/arch/x86/domain_page.c
>> @@ -51,7 +51,7 @@ static inline struct vcpu *mapcache_current_vcpu(void)
>>  if ( (v = idle_vcpu[smp_processor_id()]) == current )
>>  sync_local_execstate();
>>  /* We must now be running on the idle page table. */
>> -ASSERT(read_cr3() == __pa(idle_pg_table));
>> +ASSERT((read_cr3() & ~X86_CR3_PCID_MASK) == __pa(idle_pg_table));
> 
> This would better use X86_CR3_ADDR_MASK as well.

Right.

> 
>> @@ -102,7 +103,21 @@ void write_cr3_cr4(unsigned long cr3, unsigned long cr4)
>>  t = pre_flush();
>>  
>>  if ( read_cr4() & X86_CR4_PGE )
>> +/*
>> + * X86_CR4_PGE set means PCID being inactive.
>> + * We have to purge the TLB via flipping cr4.pge.
>> + */
>>  write_cr4(cr4 & ~X86_CR4_PGE);
>> +else if ( cpu_has_invpcid )
>> +/*
>> + * If we are using PCID purge the TLB via INVPCID as loading cr3
>> + * will affect the current PCID only.
> 
> s/current/new/ ?

Okay.

> 
>> + * If INVPCID is not supported we don't use PCIDs so loading cr3
>> + * will purge the TLB (we are in the "global pages off" branch).
>> + * invpcid_flush_all_nonglobals() seems to be faster than
>> + * invpcid_flush_all().
>> + */
>> +invpcid_flush_all_nonglobals();
>>  
>>  asm volatile ( "mov %0, %%cr3" : : "r" (cr3) : "memory" );
> 
> What about the TLB entries that have been re-created between
> the INVPCID and the write of CR3? It's not obvious to me that
> leaving them around is not going to be a problem subsequently,
> the more that you write cr3 frequently with the no-flush bit set.

The no-flush bit should not make any difference here. It controls only
flushing of TLB-entries with the new PCID.

> Don't you need to do a single context invalidation for the prior
> PCID (if different from the new one)?

Hmm, I don't think so, as the mov to %cr3 is a serializing instruction
acting as a speculation barrier. So the only TLB entries which could
survive would be the ones for the few instruction bytes after the
invpcid instruction until the end of the mov to %cr3. Those are harmless
as they are associated with the hypervisor PCID value, so there is no
risk of any data leak to a guest. Maybe a comment explaining that would
be a good idea.

> 
>> @@ -499,7 +500,26 @@ void free_shared_domheap_page(struct page_info *page)
>>  
>>  void make_cr3(struct vcpu *v, mfn_t mfn)
>>  {
>> +struct domain *d = v->domain;
>> +
>>  v->arch.cr3 = mfn_x(mfn) << PAGE_SHIFT;
>> +if ( is_pv_domain(d) && d->arch.pv_domain.pcid )
>> +v->arch.cr3 |= get_pcid_bits(v, false);
>> +}
>> +
>> +unsigned long pv_guest_cr4_to_real_cr4(struct vcpu *v)
> 
> const

Okay (for the other cases, too).

> 
>> +{
>> +struct domain *d = v->domain;
> 
> again
> 
>> @@ -298,11 +362,21 @@ int pv_domain_initialise(struct domain *d)
>>  
>>  static void _toggle_guest_pt(struct vcpu *v, bool force_cr3)
>>  {
>> +struct domain *d = v->domain;
> 
> and one more
> 
>> --- a/xen/include/asm-x86/x86-defns.h
>> +++ b/xen/include/asm-x86/x86-defns.h
>> @@ -45,7 +45,9 @@
>>  /*
>>   * Intel CPU flags in CR3
>>   */
>> -#define X86_CR3_NOFLUSH (_AC(1, ULL) << 63)
>> +#define X86_CR3_NOFLUSH(_AC(1, ULL) << 63)
>> +#define X86_CR3_ADDR_MASK  (PAGE_MASK & ~X86_CR3_NOFLUSH)
> 
> This would better be PAGE_MASK & PADDR_MASK: bits 52...62
> are reserved (the respective figure in chapter 2 of the SDM is not to
> be trusted, the tables in the "4-level paging" section are more likely to
> be correct).

Okay.


Juergen


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] seabios regression in staging

[Wei Liu]

On Thu, Mar 29, 2018 at 04:53:41PM +0200, Olaf Hering wrote:
> On Thu, Mar 29, Wei Liu wrote:
> 
> > Do you use a non-default seabios configuration? Osstest seems to be
> > happy with the update.
> 
> Not sure how I would create a non-default seabios or toolchain build.
> 
> osstest does not use SLE11, so it can not possibly spot such compile
> errors. It would certainly be cool if there would be a test with very
> old and very new toolchain.
> 

My reading of the log was that the code was actually wrong. But looking
at the code itself it should be a problem with older toolchain.

There is work on-going to use docker to test build Xen. See
.gitlab-ci.yml in xen.git. I think it would be helpful to add SLES 11 to
the list, too.

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 32/39] ARM: new VGIC: Implement arch_move_irqs()

[Andre Przywara]

Hi,

On 28/03/18 19:47, Stefano Stabellini wrote:
> On Wed, 21 Mar 2018, Andre Przywara wrote:
>> When a VCPU moves to another CPU, we need to adjust the target affinity
>> of any hardware mapped vIRQs, to observe our "physical-follows-virtual"
>> policy.
>> Implement arch_move_irqs() to adjust the physical affinity of all hardware
>> mapped vIRQs targetting this VCPU.
>>
>> Signed-off-by: Andre Przywara 
>> Reviewed-by: Julien Grall 
>> ---
>>  xen/arch/arm/vgic/vgic.c | 39 +++
>>  1 file changed, 39 insertions(+)
>>
>> diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
>> index ffab0b2635..23b8abfc5e 100644
>> --- a/xen/arch/arm/vgic/vgic.c
>> +++ b/xen/arch/arm/vgic/vgic.c
>> @@ -791,6 +791,45 @@ void gic_dump_vgic_info(struct vcpu *v)
>>  spin_unlock_irqrestore(>arch.vgic.ap_list_lock, flags);
>>  }
>>  
>> +/**
>> + * arch_move_irqs() - migrate the physical affinity of hardware mapped vIRQs
>> + * @v:  the vCPU, already assigned to the new pCPU
>> + *
>> + * arch_move_irqs() updates the physical affinity of all virtual IRQs
>> + * targetting this given vCPU. This only affects hardware mapped IRQs. The
>> + * new pCPU to target is already set in v->processor.
>> + * This is called by the core code after a vCPU has been migrated to a new
>> + * physical CPU.
>> + */
>> +void arch_move_irqs(struct vcpu *v)
>> +{
>> +struct domain *d = v->domain;
>> +unsigned int i;
>> +
>> +/* We only target SPIs with this function */
>> +for ( i = 0; i < d->arch.vgic.nr_spis; i++ )
>> +{
>> +struct vgic_irq *irq = vgic_get_irq(d, NULL, i + 
>> VGIC_NR_PRIVATE_IRQS);
>> +unsigned long flags;
>> +
>> +if ( !irq )
>> +continue;
>> +
>> +spin_lock_irqsave(>irq_lock, flags);
>> +
>> +/* only vIRQs that are not on a vCPU yet , but targetting this vCPU 
>> */
>> +if ( irq->hw && !irq->vcpu && irq->target_vcpu == v)
>> +{
> 
> In vgic_mmio_write_target, we change the physical irq affinity
> immediately, without checking for !irq->vcpu.

> I think it is OK because if a second interrupt for vcpuB comes in cpuB
> while it is still injected in vcpuA/cpuA, vgic_get_irq returns the same
> vgic_irq instance, vgic_inject_irq sets pending_latch to true.
> vgic_queue_irq_unlock does nothing because irq->vcpu is set. Then when
> vcpuA traps into Xen, vgic_prune_ap_list will take care of moving the
> vgic_irq to the ap_list belonging to vcpuB.
> 
> This seems to work, but don't we also need a vcpu_kick at the end of
> vgic_prune_ap_list to make sure the changes take effect in vcpuB? vcpuB
> could take an very long time to trap back into Xen again.

That seems like a valid question to me. But as KVM should be in the same
situation and there is no kick() there, I would like to forward this
question to Christoffer and Marc - who will probably not answer before
Tuesday. So I would ask for a followup fix if this is considered legitimate.

> But the real question is: why do we need to check for !irq->vcpu here?
> And worse: if an interrupt has irq->vcpu set, then who will take care of
> fixing the physical irq affinity later?

I think you are right, we don't need to consider irq->vcpu here. Similar
to what we now emulate, the affinity is only of concern for the *next*
IRQ to trigger. Currently handled IRQs are not affected, and a changed
affinity will not change anything for them.

> It looks like we should remove the "&& !irq->vcpu" here so that we can
> rely on the same mechanism already in place for ITARGETSR changes.
> However, would that work with already active interrupts? I think it
> should but I wanted to double check.

Looks all right to me, I will remove the "&& !irq->vcpu".

Cheers,
Andre.

>> +irq_desc_t *desc = irq_to_desc(irq->hwintid);
>> +
>> +irq_set_affinity(desc, cpumask_of(v->processor));
>> +}
>> +
>> +spin_unlock_irqrestore(>irq_lock, flags);
>> +vgic_put_irq(d, irq);
>> +}
>> +}
>> +
>>  struct irq_desc *vgic_get_hw_irq_desc(struct domain *d, struct vcpu *v,
>>unsigned int virq)
>>  {

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] seabios regression in staging

[Olaf Hering]

On Thu, Mar 29, Wei Liu wrote:

> Do you use a non-default seabios configuration? Osstest seems to be
> happy with the update.

Not sure how I would create a non-default seabios or toolchain build.

osstest does not use SLE11, so it can not possibly spot such compile
errors. It would certainly be cool if there would be a test with very
old and very new toolchain.

Olaf


signature.asc
Description: PGP signature
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] seabios regression in staging

[Wei Liu]

On Thu, Mar 29, 2018 at 01:42:43PM +0200, Olaf Hering wrote:
> It seems the latest seabios that was pulled into staging recently fails
> to compile, at least in SLE_11:
> 
> [   86s]   Compile checking out/src/hw/blockcmd.o
> [   86s] src/hw/blockcmd.c: In function 'scsi_rep_luns_scan':
> [   86s] src/hw/blockcmd.c:229: error: unknown field 'cdbcmd' specified in 
> initializer
> [   86s] src/hw/blockcmd.c:229: warning: missing braces around initializer
> [   86s] src/hw/blockcmd.c:229: warning: (near initialization for 
> 'op.')
> [   86s] src/hw/blockcmd.c:229: warning: initialization makes integer from 
> pointer without a cast
> [   86s] make[7]: *** [out/src/hw/blockcmd.o] Error 1
> 
> I can eventually trick the build to use gcc48 not only for xen but also
> for tools, but I wonder if there is a way to fix this.
> 
> Upstream may have not fixed that yet:
> https://github.com/coreboot/seabios/blame/master/src/block.h
> https://github.com/coreboot/seabios/blame/master/src/hw/blockcmd.c
> 

I think this is a problem with seabios upstream. We should ask them to
fix it and do another release.

Do you use a non-default seabios configuration? Osstest seems to be
happy with the update.

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Cut-off date for Xen 4.11 is March 30th, 2018

[Wei Liu]

On Thu, Mar 29, 2018 at 06:25:06AM -0600, Jan Beulich wrote:
> >>> On 29.03.18 at 12:50,  wrote:
> > On Thu, Mar 29, 2018 at 04:35:27AM -0600, Jan Beulich wrote:
> >> >>> On 29.03.18 at 12:22,  wrote:
> >> > On 03/29/2018 10:56 AM, Juergen Gross wrote:
> >> >> On 29/03/18 11:53, George Dunlap wrote:
> >> >>> On 03/29/2018 07:52 AM, Juergen Gross wrote:
> >>  Hi all,
> >> 
> >>  The cut-off date for Xen 4.11 is March 30th, 2018. If you want your
> >>  features to be included for the release, please make sure they are
> >>  committed by March 30th, 2018.
> >> >>>
> >> >>> March 30th is a public holiday here in the UK.  Is it the same in
> >> >>> Germany?  Would it be OK to say that things sent on Friday can be
> >> >>> committed on Tuesday 3 April if the appropriate maintainer wasn't 
> >> >>> around
> >> >>> to review them?
> >> >>>
> >> >>> If not we should warn people to get their stuff reviewed today if at 
> >> >>> all
> >> >>> possible.
> >> >>>
> >> >>> As it happens I'll be working Friday so I can check in stuff that's got
> >> >>> the right Acks / R-b's; but I won't do last-pass reviews on behalf of
> >> >>> maintainers.
> >> >> 
> >> >> I already thought of shifting the freeze by one week. Main reason is
> >> >> that several maintainers seem to have a backlog of patches to review
> >> >> which IMO should make it into 4.11.
> >> >> 
> >> >> Thoughts?
> >> > 
> >> > Well there's a benefit to this, but also a risk: that people will begin
> >> > to see the "hard freeze" as more like a "soft freeze", that will always
> >> > be pushed back / flexed if you push hard enough.  Part of the purpose of
> >> > setting the hard freeze months in advance is so that people can plan
> >> > ahead and get stuff reviewed in time; part of the reason for having
> >> > 6-month releases is so that the cost of delaying a feature / patchset to
> >> > the next release isn't very high.
> >> 
> >> As mentioned before I think anyway that we should revisit this
> >> hard freeze date approach. I would much favor a hard freeze
> >> date on where it is determined which features are intended to
> >> make it and which not. Right now at least everything related to
> >> Spectre and Meltdown would imo want to go into the category
> >> of "we'll wait until it's in".
> > 
> > You're mixing up two things: features and security fixes (and their
> > subsequent patches). I agree the latter should get special attention
> > because missing those would essentially render a release useless or
> > unattractive.  Meltdown and Spectre fall into the second category, as
> > with all the XSAs.
> 
> Subsequent patches to security fixes, unless they fix bugs in the
> earlier changes, are like feature patches to me. We're not adding
> "new functionality" as George has put it, but only want to recover
> some performance. And the switch to use INVPCID for flushing
> was intended to be done independent of XPTI. So this very much
> is a feature to me, instead of a bug fix.
> 
> Whether recovering performance is to be considered integral part
> of earlier changes causing a loss of performance (especially when
> that loss was expected) is an open question.
> 

It depends. I would say a 30-50% performance loss with XPTI will render
this release useless or unattractive to x86 users, I would argue it is
imperative to gain at least some performance back.  You and Andrew
probably have a good idea when we should call that done for this
release, whether it is just your improvement series or we also need
invpcid.

Wei.

> Jan
> 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v4 4/7] xen/x86: use invpcid for flushing the TLB

[Juergen Gross]

On 29/03/18 15:44, Jan Beulich wrote:
 On 27.03.18 at 11:07,  wrote:
>> --- a/xen/arch/x86/setup.c
>> +++ b/xen/arch/x86/setup.c
>> @@ -63,6 +63,10 @@ boolean_param("nosmp", opt_nosmp);
>>  static unsigned int __initdata max_cpus;
>>  integer_param("maxcpus", max_cpus);
>>  
>> +/* opt_invpcid: If false, don't use INVPCID instruction even if available. 
>> */
>> +static bool __initdata opt_invpcid = true;
>> +boolean_param("invpcid", opt_invpcid);
> 
> Hmm, I'm sorry for noticing only now (while seeing the questionable
> uses of cpu_has_invpcid in patch 7), but this being an init-only
> variable and having ...
> 
>> @@ -1549,6 +1553,9 @@ void __init noreturn __start_xen(unsigned long mbi_p)
>>  if ( cpu_has_fsgsbase )
>>  set_in_cr4(X86_CR4_FSGSBASE);
>>  
>> +if ( !opt_invpcid )
>> +setup_clear_cpu_cap(X86_FEATURE_INVPCID);
> 
> ... this effect has two issues: For one, in such a case this should
> be a sub-option to "cpuid=". And then afaict it also disables use of
> INVPCID in HVM guests. IOW I think you want to retain the option
> but make the variable non-init and non-static. Obviously for early
> boot use it may then no longer be possible to set it to true at build
> time (you may end up needing two variables).

Okay. I'll change it.


Juergen

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v18 05/11] x86/mm: add HYPERVISOR_memory_op to acquire guest resources

[Jan Beulich]

>>> On 29.03.18 at 15:17,  wrote:
>>  -Original Message-
>> From: Xen-devel [mailto:xen-devel-boun...@lists.xenproject.org] On Behalf
>> Of Paul Durrant
>> Sent: 29 March 2018 13:43
>> To: 'Jan Beulich' 
>> Cc: StefanoStabellini ; Wei Liu
>> ; Andrew Cooper ; Tim
>> (Xen.org) ; George Dunlap ;
>> Julien Grall ; xen-devel@lists.xenproject.org; Ian
>> Jackson 
>> Subject: Re: [Xen-devel] [PATCH v18 05/11] x86/mm: add
>> HYPERVISOR_memory_op to acquire guest resources
>> 
>> > -Original Message-
>> > From: Jan Beulich [mailto:jbeul...@suse.com]
>> > Sent: 29 March 2018 13:29
>> > To: Paul Durrant 
>> > Cc: Julien Grall ; Andrew Cooper
>> > ; George Dunlap
>> > ; Ian Jackson ; Wei
>> Liu
>> > ; StefanoStabellini ; xen-
>> > de...@lists.xenproject.org; Konrad Rzeszutek Wilk
>> > ; Tim (Xen.org) 
>> > Subject: RE: [PATCH v18 05/11] x86/mm: add HYPERVISOR_memory_op to
>> > acquire guest resources
>> >
>> > >>> On 29.03.18 at 11:53,  wrote:
>> > >> From: Jan Beulich [mailto:jbeul...@suse.com]
>> > >> Sent: 26 March 2018 12:41
>> > >>
>> > >> >>> On 22.03.18 at 12:55,  wrote:
>> > >> > --- a/xen/include/xlat.lst
>> > >> > +++ b/xen/include/xlat.lst
>> > >> > @@ -86,6 +86,7 @@
>> > >> >  !memory_map  memory.h
>> > >> >  !memory_reservation  memory.h
>> > >> >  !mem_access_op   memory.h
>> > >> > +!mem_acquire_resourcememory.h
>> > >>
>> > >> Why ! ? The layout doesn't appear to differ between native and
>> > >> compat. Or wait, the handle does, but why is that not
>> > >> XEN_GUEST_HANDLE_64()? (I've skipped the compat layer code
>> > >> in this round of review for that reason.)
>> > >
>> > > It's been XEN_GUEST_HANDLE throughout all but the earliest revisions of
>> > the
>> > > patch and I have not modified the compat code massively since you gave
>> > your
>> > > R-b anyway... the only thing that changed was copying back the new flags
>> > > value.
>> >
>> > Granted I could/should have noticed this earlier, but being able to
>> > get away without compat translation would certainly be a win, and
>> > we have that option since this is a tools-only interface.
>> >
>> 
>> Ok. I'll see if I can get this done today then.
>> 
>>   Paul
> 
> Actually, I'm getting confused by all this... The handle is for an array of 
> xen_pfn_t, which means they are going to be 32-bits wide for a 32-bit tools 
> domain. Doesn't this mean I'm going to need compat code to iterate and 
> translate the array anyway?

Oh, yes, indeed. I'm sorry for the confusion. With the other remarks
addressed feel free to add
Reviewed-by: Jan Beulich 

Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v4 7/7] xen/x86: use PCID feature

[Jan Beulich]

>>> On 27.03.18 at 11:07,  wrote:
> --- a/xen/arch/x86/domain_page.c
> +++ b/xen/arch/x86/domain_page.c
> @@ -51,7 +51,7 @@ static inline struct vcpu *mapcache_current_vcpu(void)
>  if ( (v = idle_vcpu[smp_processor_id()]) == current )
>  sync_local_execstate();
>  /* We must now be running on the idle page table. */
> -ASSERT(read_cr3() == __pa(idle_pg_table));
> +ASSERT((read_cr3() & ~X86_CR3_PCID_MASK) == __pa(idle_pg_table));

This would better use X86_CR3_ADDR_MASK as well.

> @@ -102,7 +103,21 @@ void write_cr3_cr4(unsigned long cr3, unsigned long cr4)
>  t = pre_flush();
>  
>  if ( read_cr4() & X86_CR4_PGE )
> +/*
> + * X86_CR4_PGE set means PCID being inactive.
> + * We have to purge the TLB via flipping cr4.pge.
> + */
>  write_cr4(cr4 & ~X86_CR4_PGE);
> +else if ( cpu_has_invpcid )
> +/*
> + * If we are using PCID purge the TLB via INVPCID as loading cr3
> + * will affect the current PCID only.

s/current/new/ ?

> + * If INVPCID is not supported we don't use PCIDs so loading cr3
> + * will purge the TLB (we are in the "global pages off" branch).
> + * invpcid_flush_all_nonglobals() seems to be faster than
> + * invpcid_flush_all().
> + */
> +invpcid_flush_all_nonglobals();
>  
>  asm volatile ( "mov %0, %%cr3" : : "r" (cr3) : "memory" );

What about the TLB entries that have been re-created between
the INVPCID and the write of CR3? It's not obvious to me that
leaving them around is not going to be a problem subsequently,
the more that you write cr3 frequently with the no-flush bit set.
Don't you need to do a single context invalidation for the prior
PCID (if different from the new one)?

> @@ -499,7 +500,26 @@ void free_shared_domheap_page(struct page_info *page)
>  
>  void make_cr3(struct vcpu *v, mfn_t mfn)
>  {
> +struct domain *d = v->domain;
> +
>  v->arch.cr3 = mfn_x(mfn) << PAGE_SHIFT;
> +if ( is_pv_domain(d) && d->arch.pv_domain.pcid )
> +v->arch.cr3 |= get_pcid_bits(v, false);
> +}
> +
> +unsigned long pv_guest_cr4_to_real_cr4(struct vcpu *v)

const

> +{
> +struct domain *d = v->domain;

again

> @@ -298,11 +362,21 @@ int pv_domain_initialise(struct domain *d)
>  
>  static void _toggle_guest_pt(struct vcpu *v, bool force_cr3)
>  {
> +struct domain *d = v->domain;

and one more

> --- a/xen/include/asm-x86/x86-defns.h
> +++ b/xen/include/asm-x86/x86-defns.h
> @@ -45,7 +45,9 @@
>  /*
>   * Intel CPU flags in CR3
>   */
> -#define X86_CR3_NOFLUSH (_AC(1, ULL) << 63)
> +#define X86_CR3_NOFLUSH(_AC(1, ULL) << 63)
> +#define X86_CR3_ADDR_MASK  (PAGE_MASK & ~X86_CR3_NOFLUSH)

This would better be PAGE_MASK & PADDR_MASK: bits 52...62
are reserved (the respective figure in chapter 2 of the SDM is not to
be trusted, the tables in the "4-level paging" section are more likely to
be correct).

Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v5 1/1] drm/xen-front: Add support for Xen PV display frontend

[kbuild test robot]

Hi Oleksandr,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on drm/drm-next]
[also build test ERROR on next-20180329]
[cannot apply to v4.16-rc7]
[if your patch is applied to the wrong git tree, please drop us a note to help 
improve the system]

url:
https://github.com/0day-ci/linux/commits/Oleksandr-Andrushchenko/drm-xen-front-Add-support-for-Xen-PV-display-frontend/20180329-191740
base:   git://people.freedesktop.org/~airlied/linux.git drm-next
config: x86_64-allmodconfig (attached as .config)
compiler: gcc-7 (Debian 7.3.0-1) 7.3.0
reproduce:
# save the attached .config to linux build tree
make ARCH=x86_64 

All errors (new ones prefixed by >>):

   drivers/gpu/drm/xen/xen_drm_front.c:484:13: sparse: undefined identifier 
'drm_dev_enter'
   drivers/gpu/drm/xen/xen_drm_front.c:487:17: sparse: undefined identifier 
'drm_dev_exit'
   drivers/gpu/drm/xen/xen_drm_front.c:484:26: sparse: call with no type!
   drivers/gpu/drm/xen/xen_drm_front.c:487:29: sparse: call with no type!
   drivers/gpu/drm/xen/xen_drm_front.c: In function 
'xen_drm_drv_free_object_unlocked':
>> drivers/gpu/drm/xen/xen_drm_front.c:484:6: error: implicit declaration of 
>> function 'drm_dev_enter'; did you mean 'drm_dev_unref'? 
>> [-Werror=implicit-function-declaration]
 if (drm_dev_enter(obj->dev, )) {
 ^
 drm_dev_unref
>> drivers/gpu/drm/xen/xen_drm_front.c:487:3: error: implicit declaration of 
>> function 'drm_dev_exit'; did you mean 'drm_dev_init'? 
>> [-Werror=implicit-function-declaration]
  drm_dev_exit(idx);
  ^~~~
  drm_dev_init
   cc1: some warnings being treated as errors
--
   drivers/gpu/drm/xen/xen_drm_front_kms.c:40:13: sparse: undefined identifier 
'drm_dev_enter'
   drivers/gpu/drm/xen/xen_drm_front_kms.c:43:17: sparse: undefined identifier 
'drm_dev_exit'
   drivers/gpu/drm/xen/xen_drm_front_kms.c:119:14: sparse: undefined identifier 
'drm_dev_enter'
   drivers/gpu/drm/xen/xen_drm_front_kms.c:132:9: sparse: undefined identifier 
'drm_dev_exit'
   drivers/gpu/drm/xen/xen_drm_front_kms.c:141:13: sparse: undefined identifier 
'drm_dev_enter'
   drivers/gpu/drm/xen/xen_drm_front_kms.c:144:17: sparse: undefined identifier 
'drm_dev_exit'
   drivers/gpu/drm/xen/xen_drm_front_kms.c:258:14: sparse: undefined identifier 
'drm_dev_enter'
   drivers/gpu/drm/xen/xen_drm_front_kms.c:274:9: sparse: undefined identifier 
'drm_dev_exit'
   drivers/gpu/drm/xen/xen_drm_front_kms.c:296:19: sparse: incorrect type in 
initializer (different argument counts)
   drivers/gpu/drm/xen/xen_drm_front_kms.c:296:19:expected void ( *enable 
)( ... )
   drivers/gpu/drm/xen/xen_drm_front_kms.c:296:19:got void ( * )( 
... )
   drivers/gpu/drm/xen/xen_drm_front_kms.c:40:26: sparse: call with no type!
   drivers/gpu/drm/xen/xen_drm_front_kms.c:43:29: sparse: call with no type!
   drivers/gpu/drm/xen/xen_drm_front_kms.c:119:27: sparse: call with no type!
   drivers/gpu/drm/xen/xen_drm_front_kms.c:132:21: sparse: call with no type!
   drivers/gpu/drm/xen/xen_drm_front_kms.c:141:26: sparse: call with no type!
   drivers/gpu/drm/xen/xen_drm_front_kms.c:144:29: sparse: call with no type!
   drivers/gpu/drm/xen/xen_drm_front_kms.c:258:27: sparse: call with no type!
   drivers/gpu/drm/xen/xen_drm_front_kms.c:274:21: sparse: call with no type!
   drivers/gpu/drm/xen/xen_drm_front_kms.c: In function 'fb_destroy':
>> drivers/gpu/drm/xen/xen_drm_front_kms.c:40:6: error: implicit declaration of 
>> function 'drm_dev_enter'; did you mean 'drm_dev_unref'? 
>> [-Werror=implicit-function-declaration]
 if (drm_dev_enter(fb->dev, )) {
 ^
 drm_dev_unref
>> drivers/gpu/drm/xen/xen_drm_front_kms.c:43:3: error: implicit declaration of 
>> function 'drm_dev_exit'; did you mean 'drm_dev_init'? 
>> [-Werror=implicit-function-declaration]
  drm_dev_exit(idx);
  ^~~~
  drm_dev_init
   drivers/gpu/drm/xen/xen_drm_front_kms.c: At top level:
   drivers/gpu/drm/xen/xen_drm_front_kms.c:296:12: error: initialization from 
incompatible pointer type [-Werror=incompatible-pointer-types]
 .enable = display_enable,
   ^~
   drivers/gpu/drm/xen/xen_drm_front_kms.c:296:12: note: (near initialization 
for 'display_funcs.enable')
   cc1: some warnings being treated as errors

sparse warnings: (new ones prefixed by >>)

   drivers/gpu/drm/xen/xen_drm_front.c:484:13: sparse: undefined identifier 
'drm_dev_enter'
   drivers/gpu/drm/xen/xen_drm_front.c:487:17: sparse: undefined identifier 
'drm_dev_exit'
>> drivers/gpu/drm/xen/xen_drm_front.c:484:26: sparse: call with no type!
   drivers/gpu/drm/xen/xen_drm_front.c:487:29: sparse: call with no type!
   drivers/gpu/drm/xen/xen_drm_front.c: In function 
'xen_drm_drv_free_object_unlocked':
   drivers/gpu/drm/xen/xen_drm_front.c:484:6: error: implicit declaration of 

Re: [Xen-devel] [PATCH v4 4/7] xen/x86: use invpcid for flushing the TLB

[Jan Beulich]

>>> On 27.03.18 at 11:07,  wrote:
> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -63,6 +63,10 @@ boolean_param("nosmp", opt_nosmp);
>  static unsigned int __initdata max_cpus;
>  integer_param("maxcpus", max_cpus);
>  
> +/* opt_invpcid: If false, don't use INVPCID instruction even if available. */
> +static bool __initdata opt_invpcid = true;
> +boolean_param("invpcid", opt_invpcid);

Hmm, I'm sorry for noticing only now (while seeing the questionable
uses of cpu_has_invpcid in patch 7), but this being an init-only
variable and having ...

> @@ -1549,6 +1553,9 @@ void __init noreturn __start_xen(unsigned long mbi_p)
>  if ( cpu_has_fsgsbase )
>  set_in_cr4(X86_CR4_FSGSBASE);
>  
> +if ( !opt_invpcid )
> +setup_clear_cpu_cap(X86_FEATURE_INVPCID);

... this effect has two issues: For one, in such a case this should
be a sub-option to "cpuid=". And then afaict it also disables use of
INVPCID in HVM guests. IOW I think you want to retain the option
but make the variable non-init and non-static. Obviously for early
boot use it may then no longer be possible to set it to true at build
time (you may end up needing two variables).

Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 06/39] ARM: evtchn: Handle level triggered IRQs correctly

[Andre Przywara]

Hi,

On 28/03/18 18:46, Stefano Stabellini wrote:
> On Wed, 28 Mar 2018, Andre Przywara wrote:
>> On 28/03/18 01:01, Stefano Stabellini wrote:
>>> On Wed, 21 Mar 2018, Andre Przywara wrote:
 The event channel IRQ has level triggered semantics, however the current
 VGIC treats everything as edge triggered.
 To correctly process those IRQs, we have to lower the (virtual) IRQ line
 at some point in time, depending on whether ther interrupt condition
 still prevails.
 Check the per-VCPU evtchn_upcall_pending variable to make the interrupt
 line match its status, and call this function upon every hypervisor
 entry.

 Signed-off-by: Andre Przywara 
 Reviewed-by: Julien Grall 
 ---
  xen/arch/arm/domain.c   | 7 +++
  xen/arch/arm/traps.c| 1 +
  xen/include/asm-arm/event.h | 1 +
  3 files changed, 9 insertions(+)

 diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
 index ff97f2bc76..9688e62f78 100644
 --- a/xen/arch/arm/domain.c
 +++ b/xen/arch/arm/domain.c
 @@ -953,6 +953,13 @@ void vcpu_mark_events_pending(struct vcpu *v)
  vgic_inject_irq(v->domain, v, v->domain->arch.evtchn_irq, true);
  }
  
 +void vcpu_update_evtchn_irq(struct vcpu *v)
 +{
 +bool pending = vcpu_info(v, evtchn_upcall_pending);
 +
 +vgic_inject_irq(v->domain, v, v->domain->arch.evtchn_irq, pending);
 +}
 +
  /* The ARM spec declares that even if local irqs are masked in
   * the CPSR register, an irq should wake up a cpu from WFI anyway.
   * For this reason we need to check for irqs that need delivery,
 diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
 index 2638446693..5c18e918b0 100644
 --- a/xen/arch/arm/traps.c
 +++ b/xen/arch/arm/traps.c
 @@ -2033,6 +2033,7 @@ static void enter_hypervisor_head(struct 
 cpu_user_regs *regs)
   * trap and how it can be optimised.
   */
  vtimer_update_irqs(current);
 +vcpu_update_evtchn_irq(current);
  #endif
>>>
>>> I am replying to this patch, even though I have already committed it, to
>>> point out a problem with the way we currently handle the evtchn_irq in
>>> this series.
>>>
>>> The short version is that I think we should configure the PPI
>>> corresponding to the evtchn_irq as EDGE instead of LEVEL.
>>
>> Well, that's really a separate problem, then. We can't just configure
>> the PPI at will, it has to match the device semantic.
>> When writing this patch, I checked how the the evtchn "device" is
>> implemented, and it screams "level IRQ" to me:
>> - We have a flag (evtchn_upcall_pending), which stores the current
>> interrupt state.
>> - This flag gets set by the producer when the interrupt condition is true.
>> - It gets cleared by the *consumer* once it has handled the request.
>>
>> So if the event channel mechanism should be edge (which would be fair
>> enough), we need to change the code to implement this: the interrupt
>> condition should be cleared once we *injected* the IRQ - and not only
>> when the consumer has signalled completion.
>>
>> Another thing to consider: by the spec the *configurability* of PPIs is
>> implementation defined. The KVM implementation chose to fix all of them
>> to "level", which we need for the arch timer. So setting the evtchn PPI
>> to edge would be ignored. We could deviate from that, but I need to
>> check what the side effects are.
>>
>>> The long explanation follows, please correct me if I am wrong.
>>>
>>> 1) vcpuA/cpuA is running, it has already handled the event, cleared
>>> evtchn_upcall_pending and EOIed the event_irq but hasn't trapped into
>>> Xen yet. It is still in guest mode.
>>>
>>> 2) Xen on cpuB calls vcpu_mark_events_pending(vcpuA), then calls
>>> vgic_inject_irq. However, because irq->line_level is high, it is not
>>> injected.

I was just wondering if we could do something similar to the SBSA UART
change, where we amend vcpu_mark_events_pending() to update the line
level in the VGIC via vgic_inject_irq().

>> So this is a case where we fail to sync in time on the actual emulated
>> line level.

So to explain this:
The virtual line level is not in sync all of the time, as we don't get
signalled when the flag is cleared by the domain. We need to sync this
whenever needed, which, for once, is when the domain returns to the
hypervisor. Another point is this vcpu_mark_events_pending() here,
because we need to know the state. And fortunately we just queried it
also. So I will include a change which updates this into the new VGIC.

 KVM recently gained some nice code to solve this: We can
>> register per-IRQ functions that return the line level. For hardware
>> mapped IRQs this queries the distributor, but for the arch timer for
>> instance it just uses a shortcut to read CNTV_CTL_EL0.
>> The evtchn IRQ could just check 

[Xen-devel] [PATCH 0/1] drm/xen-zcopy: Add Xen zero-copy helper DRM driver

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

Hello!

When using Xen PV DRM frontend driver then on backend side one will need
to do copying of display buffers' contents (filled by the
frontend's user-space) into buffers allocated at the backend side.
Taking into account the size of display buffers and frames per seconds
it may result in unneeded huge data bus occupation and performance loss.

This helper driver allows implementing zero-copying use-cases
when using Xen para-virtualized frontend display driver by
implementing a DRM/KMS helper driver running on backend's side.
It utilizes PRIME buffers API to share frontend's buffers with
physical device drivers on backend's side:

 - a dumb buffer created on backend's side can be shared
   with the Xen PV frontend driver, so it directly writes
   into backend's domain memory (into the buffer exported from
   DRM/KMS driver of a physical display device)
 - a dumb buffer allocated by the frontend can be imported
   into physical device DRM/KMS driver, thus allowing to
   achieve no copying as well

For that reason number of IOCTLs are introduced:
 -  DRM_XEN_ZCOPY_DUMB_FROM_REFS
This will create a DRM dumb buffer from grant references provided
by the frontend
 - DRM_XEN_ZCOPY_DUMB_TO_REFS
   This will grant references to a dumb/display buffer's memory provided
   by the backend
 - DRM_XEN_ZCOPY_DUMB_WAIT_FREE
   This will block until the dumb buffer with the wait handle provided
   be freed

With this helper driver I was able to drop CPU usage from 17% to 3%
on Renesas R-Car M3 board.

This was tested with Renesas' Wayland-KMS and backend running as DRM master.

Thank you,
Oleksandr

Oleksandr Andrushchenko (1):
  drm/xen-zcopy: Add Xen zero-copy helper DRM driver

 Documentation/gpu/drivers.rst   |   1 +
 Documentation/gpu/xen-zcopy.rst |  32 +
 drivers/gpu/drm/xen/Kconfig |  25 +
 drivers/gpu/drm/xen/Makefile|   5 +
 drivers/gpu/drm/xen/xen_drm_zcopy.c | 880 
 drivers/gpu/drm/xen/xen_drm_zcopy_balloon.c | 154 +
 drivers/gpu/drm/xen/xen_drm_zcopy_balloon.h |  38 ++
 include/uapi/drm/xen_zcopy_drm.h| 129 
 8 files changed, 1264 insertions(+)
 create mode 100644 Documentation/gpu/xen-zcopy.rst
 create mode 100644 drivers/gpu/drm/xen/xen_drm_zcopy.c
 create mode 100644 drivers/gpu/drm/xen/xen_drm_zcopy_balloon.c
 create mode 100644 drivers/gpu/drm/xen/xen_drm_zcopy_balloon.h
 create mode 100644 include/uapi/drm/xen_zcopy_drm.h

-- 
2.16.2


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 1/1] drm/xen-zcopy: Add Xen zero-copy helper DRM driver

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

Introduce Xen zero-copy helper DRM driver, add user-space API of the driver:
1. DRM_IOCTL_XEN_ZCOPY_DUMB_FROM_REFS
This will create a DRM dumb buffer from grant references provided
by the frontend. The intended usage is:
  - Frontend
- creates a dumb/display buffer and allocates memory
- grants foreign access to the buffer pages
- passes granted references to the backend
  - Backend
- issues DRM_XEN_ZCOPY_DUMB_FROM_REFS ioctl to map
  granted references and create a dumb buffer
- requests handle to fd conversion via DRM_IOCTL_PRIME_HANDLE_TO_FD
- requests real HW driver/consumer to import the PRIME buffer with
  DRM_IOCTL_PRIME_FD_TO_HANDLE
- uses handle returned by the real HW driver
  - at the end:
o closes real HW driver's handle with DRM_IOCTL_GEM_CLOSE
o closes zero-copy driver's handle with DRM_IOCTL_GEM_CLOSE
o closes file descriptor of the exported buffer

2. DRM_IOCTL_XEN_ZCOPY_DUMB_TO_REFS
This will grant references to a dumb/display buffer's memory provided by the
backend. The intended usage is:
  - Frontend
- requests backend to allocate dumb/display buffer and grant references
  to its pages
  - Backend
- requests real HW driver to create a dumb with DRM_IOCTL_MODE_CREATE_DUMB
- requests handle to fd conversion via DRM_IOCTL_PRIME_HANDLE_TO_FD
- requests zero-copy driver to import the PRIME buffer with
  DRM_IOCTL_PRIME_FD_TO_HANDLE
- issues DRM_XEN_ZCOPY_DUMB_TO_REFS ioctl to
  grant references to the buffer's memory.
- passes grant references to the frontend
 - at the end:
- closes zero-copy driver's handle with DRM_IOCTL_GEM_CLOSE
- closes real HW driver's handle with DRM_IOCTL_GEM_CLOSE
- closes file descriptor of the imported buffer

Implement GEM/IOCTL handling depending on driver mode of operation:
- if GEM is created from grant references, then prepare to create
  a dumb from mapped pages
- if GEM grant references are about to be provided for the
  imported PRIME buffer, then prepare for granting references
  and providing those to user-space

Implement handling of display buffers from backend to/from front
interaction point ov view:
- when importing a buffer from the frontend:
  - allocate/free xen ballooned pages via Xen balloon driver
or by manually allocating a DMA buffer
  - if DMA buffer is used, then increase/decrease its pages
reservation accordingly
  - map/unmap foreign pages to the ballooned pages
- when exporting a buffer to the frontend:
  - grant references for the pages of the imported PRIME buffer
  - pass the grants back to user-space, so those can be shared
with the frontend

Add an option to allocate DMA buffers as backing storage while
importing a frontend's buffer into host's memory:
for those use-cases when exported PRIME buffer will be used by
a device expecting CMA buffers only, it is possible to map
frontend's pages onto contiguous buffer, e.g. allocated via
DMA API.

Implement synchronous buffer deletion: for buffers, created from front's
grant references, synchronization between backend and frontend is needed
on buffer deletion as front expects us to unmap these references after
XENDISPL_OP_DBUF_DESTROY response.
For that reason introduce DRM_IOCTL_XEN_ZCOPY_DUMB_WAIT_FREE IOCTL:
this will block until dumb buffer, with the wait handle provided,
be freed.

The rationale behind implementing own wait handle:
  - dumb buffer handle cannot be used as when the PRIME buffer
gets exported there are at least 2 handles: one is for the
backend and another one for the importing application,
so when backend closes its handle and the other application still
holds the buffer then there is no way for the backend to tell
which buffer we want to wait for while calling xen_ioctl_wait_free
  - flink cannot be used as well as it is gone when DRM core
calls .gem_free_object_unlocked

Signed-off-by: Oleksandr Andrushchenko 
---
 Documentation/gpu/drivers.rst   |   1 +
 Documentation/gpu/xen-zcopy.rst |  32 +
 drivers/gpu/drm/xen/Kconfig |  25 +
 drivers/gpu/drm/xen/Makefile|   5 +
 drivers/gpu/drm/xen/xen_drm_zcopy.c | 880 
 drivers/gpu/drm/xen/xen_drm_zcopy_balloon.c | 154 +
 drivers/gpu/drm/xen/xen_drm_zcopy_balloon.h |  38 ++
 include/uapi/drm/xen_zcopy_drm.h| 129 
 8 files changed, 1264 insertions(+)
 create mode 100644 Documentation/gpu/xen-zcopy.rst
 create mode 100644 drivers/gpu/drm/xen/xen_drm_zcopy.c
 create mode 100644 drivers/gpu/drm/xen/xen_drm_zcopy_balloon.c
 create mode 100644 drivers/gpu/drm/xen/xen_drm_zcopy_balloon.h
 create mode 100644 include/uapi/drm/xen_zcopy_drm.h

diff --git a/Documentation/gpu/drivers.rst b/Documentation/gpu/drivers.rst
index d3ab6abae838..900ff1c3d3f1 100644
--- a/Documentation/gpu/drivers.rst
+++ 

[Xen-devel] [linux-next test] 121327: regressions - FAIL

[osstest service owner]

flight 121327 linux-next real [real]
http://logs.test-lab.xenproject.org/osstest/logs/121327/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-libvirt-raw 10 debian-di-installfail REGR. vs. 121315
 build-arm64-pvops 6 kernel-build fail REGR. vs. 121315
 test-armhf-armhf-libvirt-xsm 19 leak-check/check fail REGR. vs. 121315
 test-armhf-armhf-xl-credit2  10 debian-install   fail REGR. vs. 121315
 test-amd64-amd64-qemuu-nested-intel 17 debian-hvm-install/l1/l2 fail REGR. vs. 
121315
 test-armhf-armhf-xl-xsm  10 debian-install   fail REGR. vs. 121315

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-examine  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-qemut-rhel6hvm-amd 10 redhat-installfail blocked in 121315
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop   fail blocked in 121315
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop   fail blocked in 121315
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop   fail blocked in 121315
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 121315
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail  like 121315
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 121315
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 121315
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 121315
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stopfail like 121315
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass

version targeted for testing:
 linux7373fc81dadd068a8f2ea26011774f00f1f156bd
baseline version:
 linux3eb2ce825ea1ad89d20f7a3b5780df850e4be274

Last test of basis  (not found) 
Failing since   (not found) 
Testing same since   121327  2018-03-28 10:14:18 Z1 days1 attempts

jobs:
 build-amd64-xsm  pass
 build-arm64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-arm64  pass
 build-armhf 

Re: [Xen-devel] [PATCH v18 05/11] x86/mm: add HYPERVISOR_memory_op to acquire guest resources

[Paul Durrant]

> -Original Message-
> From: Xen-devel [mailto:xen-devel-boun...@lists.xenproject.org] On Behalf
> Of Paul Durrant
> Sent: 29 March 2018 13:43
> To: 'Jan Beulich' 
> Cc: StefanoStabellini ; Wei Liu
> ; Andrew Cooper ; Tim
> (Xen.org) ; George Dunlap ;
> Julien Grall ; xen-devel@lists.xenproject.org; Ian
> Jackson 
> Subject: Re: [Xen-devel] [PATCH v18 05/11] x86/mm: add
> HYPERVISOR_memory_op to acquire guest resources
> 
> > -Original Message-
> > From: Jan Beulich [mailto:jbeul...@suse.com]
> > Sent: 29 March 2018 13:29
> > To: Paul Durrant 
> > Cc: Julien Grall ; Andrew Cooper
> > ; George Dunlap
> > ; Ian Jackson ; Wei
> Liu
> > ; StefanoStabellini ; xen-
> > de...@lists.xenproject.org; Konrad Rzeszutek Wilk
> > ; Tim (Xen.org) 
> > Subject: RE: [PATCH v18 05/11] x86/mm: add HYPERVISOR_memory_op to
> > acquire guest resources
> >
> > >>> On 29.03.18 at 11:53,  wrote:
> > >> From: Jan Beulich [mailto:jbeul...@suse.com]
> > >> Sent: 26 March 2018 12:41
> > >>
> > >> >>> On 22.03.18 at 12:55,  wrote:
> > >> > --- a/xen/include/xlat.lst
> > >> > +++ b/xen/include/xlat.lst
> > >> > @@ -86,6 +86,7 @@
> > >> >  ! memory_map  memory.h
> > >> >  ! memory_reservation  memory.h
> > >> >  ! mem_access_op   memory.h
> > >> > +! mem_acquire_resourcememory.h
> > >>
> > >> Why ! ? The layout doesn't appear to differ between native and
> > >> compat. Or wait, the handle does, but why is that not
> > >> XEN_GUEST_HANDLE_64()? (I've skipped the compat layer code
> > >> in this round of review for that reason.)
> > >
> > > It's been XEN_GUEST_HANDLE throughout all but the earliest revisions of
> > the
> > > patch and I have not modified the compat code massively since you gave
> > your
> > > R-b anyway... the only thing that changed was copying back the new flags
> > > value.
> >
> > Granted I could/should have noticed this earlier, but being able to
> > get away without compat translation would certainly be a win, and
> > we have that option since this is a tools-only interface.
> >
> 
> Ok. I'll see if I can get this done today then.
> 
>   Paul

Actually, I'm getting confused by all this... The handle is for an array of 
xen_pfn_t, which means they are going to be 32-bits wide for a 32-bit tools 
domain. Doesn't this mean I'm going to need compat code to iterate and 
translate the array anyway?

  Paul

> 
> > Jan
> 
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xenproject.org
> https://lists.xenproject.org/mailman/listinfo/xen-devel
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v4 5/7] xen/x86: disable global pages for domains with XPTI active

[Jan Beulich]

>>> On 27.03.18 at 11:07,  wrote:
> Instead of flushing the TLB from global pages when switching address
> spaces with XPTI being active just disable global pages via %cr4
> completely when a domain subject to XPTI is active. This avoids the
> need for extra TLB flushes as loading %cr3 will remove all TLB
> entries.
> 
> In order to avoid states with cr3/cr4 having inconsistent values
> (e.g. global pages being activated while cr3 already specifies a XPTI
> address space) move loading of the new cr4 value to write_ptbase()
> (actually to write_cr3_cr4() called by write_ptbase()).
> 
> Signed-off-by: Juergen Gross 

Reviewed-by: Jan Beulich 



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v18 05/11] x86/mm: add HYPERVISOR_memory_op to acquire guest resources

[Paul Durrant]

> -Original Message-
> From: Jan Beulich [mailto:jbeul...@suse.com]
> Sent: 29 March 2018 13:29
> To: Paul Durrant 
> Cc: Julien Grall ; Andrew Cooper
> ; George Dunlap
> ; Ian Jackson ; Wei Liu
> ; StefanoStabellini ; xen-
> de...@lists.xenproject.org; Konrad Rzeszutek Wilk
> ; Tim (Xen.org) 
> Subject: RE: [PATCH v18 05/11] x86/mm: add HYPERVISOR_memory_op to
> acquire guest resources
> 
> >>> On 29.03.18 at 11:53,  wrote:
> >> From: Jan Beulich [mailto:jbeul...@suse.com]
> >> Sent: 26 March 2018 12:41
> >>
> >> >>> On 22.03.18 at 12:55,  wrote:
> >> > --- a/xen/include/xlat.lst
> >> > +++ b/xen/include/xlat.lst
> >> > @@ -86,6 +86,7 @@
> >> >  !   memory_map  memory.h
> >> >  !   memory_reservation  memory.h
> >> >  !   mem_access_op   memory.h
> >> > +!   mem_acquire_resourcememory.h
> >>
> >> Why ! ? The layout doesn't appear to differ between native and
> >> compat. Or wait, the handle does, but why is that not
> >> XEN_GUEST_HANDLE_64()? (I've skipped the compat layer code
> >> in this round of review for that reason.)
> >
> > It's been XEN_GUEST_HANDLE throughout all but the earliest revisions of
> the
> > patch and I have not modified the compat code massively since you gave
> your
> > R-b anyway... the only thing that changed was copying back the new flags
> > value.
> 
> Granted I could/should have noticed this earlier, but being able to
> get away without compat translation would certainly be a win, and
> we have that option since this is a tools-only interface.
> 

Ok. I'll see if I can get this done today then.

  Paul

> Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v18 10/11] common: add a new mappable resource type: XENMEM_resource_grant_table

[Jan Beulich]

>>> On 29.03.18 at 13:02,  wrote:
>> From: Jan Beulich [mailto:jbeul...@suse.com]
>> Sent: 26 March 2018 13:55
>> 
>> >>> On 26.03.18 at 14:16,  wrote:
>>  On 22.03.18 at 12:55,  wrote:
>> >> --- a/xen/common/grant_table.c
>> >> +++ b/xen/common/grant_table.c
>> >> @@ -3863,6 +3863,35 @@ int mem_sharing_gref_to_gfn(struct
>> grant_table *gt, grant_ref_t ref,
>> >>  }
>> >>  #endif
>> >>
>> >> +/* caller must hold read or write lock */
>> >> +static int gnttab_get_status_frame_mfn(struct domain *d,
>> >> +   unsigned long idx, mfn_t *mfn)
>> >> +{
>> >> +struct grant_table *gt = d->grant_table;
>> >> +
>> >> +if ( idx >= nr_status_frames(gt) )
>> >> +return -EINVAL;
>> >> +
>> >> +*mfn = _mfn(virt_to_mfn(gt->status[idx]));
>> >> +return 0;
>> >> +}
>> >> +
>> >> +/* caller must hold write lock */
>> >> +static int gnttab_get_shared_frame_mfn(struct domain *d,
>> >> +   unsigned long idx, mfn_t *mfn)
>> >> +{
>> >> +struct grant_table *gt = d->grant_table;
>> >> +
>> >> +if ( (idx >= nr_grant_frames(gt)) && (idx < gt->max_grant_frames) )
>> >> +gnttab_grow_table(d, idx + 1);
>> >> +
>> >> +if ( idx >= nr_grant_frames(gt) )
>> >> +return -EINVAL;
>> >> +
>> >> +*mfn = _mfn(virt_to_mfn(gt->shared_raw[idx]));
>> >> +return 0;
>> >> +}
>> >
>> > I realize the anomaly was there already before, but imo it becomes
>> > more pronounced with the two functions differing in more than just
>> > the shared vs status naming (IOW I find it strange that one grows
>> > the grant table while the other doesn't). This extends to ...
>> >
>> >> +int gnttab_get_shared_frame(struct domain *d, unsigned long idx,
>> >> +mfn_t *mfn)
>> >> +{
>> >> +struct grant_table *gt = d->grant_table;
>> >> +int rc;
>> >> +
>> >> +grant_write_lock(gt);
>> >> +rc = gnttab_get_shared_frame_mfn(d, idx, mfn);
>> >> +grant_write_unlock(gt);
>> >> +
>> >> +return rc;
>> >> +}
>> >> +
>> >> +int gnttab_get_status_frame(struct domain *d, unsigned long idx,
>> >> +mfn_t *mfn)
>> >> +{
>> >> +struct grant_table *gt = d->grant_table;
>> >> +int rc;
>> >> +
>> >> +grant_read_lock(gt);
>> >> +rc = gnttab_get_status_frame_mfn(d, idx, mfn);
>> >> +grant_read_unlock(gt);
>> >> +
>> >> +return rc;
>> >> +}
>> >
>> > ... these two acquiring the lock in different ways.
> 
> So, you want me to have gnttab_get_status_frame() grow the table 
> accordingly? I'd really rather not do that at v19 of the series when it's 
> never been part of the scope before.

Indeed, please only take this as a remark at this point in time.
It just so happens that when you look at something again after
a fair amount of time, you see things you haven't noticed before.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v18 05/11] x86/mm: add HYPERVISOR_memory_op to acquire guest resources

[Jan Beulich]

>>> On 29.03.18 at 11:53,  wrote:
>> From: Jan Beulich [mailto:jbeul...@suse.com]
>> Sent: 26 March 2018 12:41
>> 
>> >>> On 22.03.18 at 12:55,  wrote:
>> > --- a/xen/include/xlat.lst
>> > +++ b/xen/include/xlat.lst
>> > @@ -86,6 +86,7 @@
>> >  ! memory_map  memory.h
>> >  ! memory_reservation  memory.h
>> >  ! mem_access_op   memory.h
>> > +! mem_acquire_resourcememory.h
>> 
>> Why ! ? The layout doesn't appear to differ between native and
>> compat. Or wait, the handle does, but why is that not
>> XEN_GUEST_HANDLE_64()? (I've skipped the compat layer code
>> in this round of review for that reason.)
> 
> It's been XEN_GUEST_HANDLE throughout all but the earliest revisions of the 
> patch and I have not modified the compat code massively since you gave your 
> R-b anyway... the only thing that changed was copying back the new flags 
> value.

Granted I could/should have noticed this earlier, but being able to
get away without compat translation would certainly be a win, and
we have that option since this is a tools-only interface.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Cut-off date for Xen 4.11 is March 30th, 2018

[Jan Beulich]

>>> On 29.03.18 at 12:50,  wrote:
> On Thu, Mar 29, 2018 at 04:35:27AM -0600, Jan Beulich wrote:
>> >>> On 29.03.18 at 12:22,  wrote:
>> > On 03/29/2018 10:56 AM, Juergen Gross wrote:
>> >> On 29/03/18 11:53, George Dunlap wrote:
>> >>> On 03/29/2018 07:52 AM, Juergen Gross wrote:
>>  Hi all,
>> 
>>  The cut-off date for Xen 4.11 is March 30th, 2018. If you want your
>>  features to be included for the release, please make sure they are
>>  committed by March 30th, 2018.
>> >>>
>> >>> March 30th is a public holiday here in the UK.  Is it the same in
>> >>> Germany?  Would it be OK to say that things sent on Friday can be
>> >>> committed on Tuesday 3 April if the appropriate maintainer wasn't around
>> >>> to review them?
>> >>>
>> >>> If not we should warn people to get their stuff reviewed today if at all
>> >>> possible.
>> >>>
>> >>> As it happens I'll be working Friday so I can check in stuff that's got
>> >>> the right Acks / R-b's; but I won't do last-pass reviews on behalf of
>> >>> maintainers.
>> >> 
>> >> I already thought of shifting the freeze by one week. Main reason is
>> >> that several maintainers seem to have a backlog of patches to review
>> >> which IMO should make it into 4.11.
>> >> 
>> >> Thoughts?
>> > 
>> > Well there's a benefit to this, but also a risk: that people will begin
>> > to see the "hard freeze" as more like a "soft freeze", that will always
>> > be pushed back / flexed if you push hard enough.  Part of the purpose of
>> > setting the hard freeze months in advance is so that people can plan
>> > ahead and get stuff reviewed in time; part of the reason for having
>> > 6-month releases is so that the cost of delaying a feature / patchset to
>> > the next release isn't very high.
>> 
>> As mentioned before I think anyway that we should revisit this
>> hard freeze date approach. I would much favor a hard freeze
>> date on where it is determined which features are intended to
>> make it and which not. Right now at least everything related to
>> Spectre and Meltdown would imo want to go into the category
>> of "we'll wait until it's in".
> 
> You're mixing up two things: features and security fixes (and their
> subsequent patches). I agree the latter should get special attention
> because missing those would essentially render a release useless or
> unattractive.  Meltdown and Spectre fall into the second category, as
> with all the XSAs.

Subsequent patches to security fixes, unless they fix bugs in the
earlier changes, are like feature patches to me. We're not adding
"new functionality" as George has put it, but only want to recover
some performance. And the switch to use INVPCID for flushing
was intended to be done independent of XPTI. So this very much
is a feature to me, instead of a bug fix.

Whether recovering performance is to be considered integral part
of earlier changes causing a loss of performance (especially when
that loss was expected) is an open question.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v6] new config option vtsc_tolerance_khz to avoid TSC emulation

[Olaf Hering]

Add an option to control when vTSC emulation will be activated for a
domU with tsc_mode=default. Without such option each TSC access from
domU will be emulated, which causes a significant perfomance drop for
workloads that make use of rdtsc.

One option to avoid the TSC option is to run domUs with tsc_mode=native.
This has the drawback that migrating a domU from a "2.3GHz" class host
to a "2.4GHz" class host may change the rate at wich the TSC counter
increases, the domU may not be prepared for that.

With the new option the host admin can decide how a domU should behave
when it is migrated across systems of the same class. Since there is
always some jitter when Xen calibrates the cpu_khz value, all hosts of
the same class will most likely have slightly different values. As a
result vTSC emulation is unavoidable. Data collected during the incident
which triggered this change showed a jitter of up to 200 KHz across
systems of the same class.

Existing padding fields are reused to store vtsc_khz_tolerance as u16.

v6:
 - mention default value in xl.cfg
 - tsc_set_info: remove usage of __func__, use %d for domid
 - tsc_set_info: use ABS to calculate khz_diff
v5:
 - reduce functionality to allow setting of the tolerance value
   only at initial domU startup
v4:
 - add missing copyback in XEN_DOMCTL_set_vtsc_tolerance_khz
v3:
 - rename vtsc_khz_tolerance to vtsc_tolerance_khz
 - separate domctls to adjust values
 - more docs
 - update libxl.h
 - update python tests
 - flask check bound to tsc permissions
 - not runtime tested due to dlsym() build errors in staging

Signed-off-by: Olaf Hering 
---
 docs/man/xen-tscmode.pod.7   | 16 
 docs/man/xl.cfg.pod.5.in | 10 ++
 docs/specs/libxc-migration-stream.pandoc |  6 --
 tools/libxc/include/xenctrl.h|  2 ++
 tools/libxc/xc_domain.c  |  4 
 tools/libxc/xc_sr_common_x86.c   |  6 --
 tools/libxc/xc_sr_stream_format.h|  3 ++-
 tools/libxl/libxl.h  |  6 ++
 tools/libxl/libxl_types.idl  |  1 +
 tools/libxl/libxl_x86.c  |  3 ++-
 tools/python/xen/lowlevel/xc/xc.c|  2 +-
 tools/xl/xl_parse.c  |  3 +++
 xen/arch/x86/domain.c|  2 +-
 xen/arch/x86/domctl.c|  2 ++
 xen/arch/x86/time.c  | 30 +++---
 xen/include/asm-x86/domain.h |  1 +
 xen/include/asm-x86/time.h   |  6 --
 xen/include/public/domctl.h  |  3 ++-
 18 files changed, 92 insertions(+), 14 deletions(-)

diff --git a/docs/man/xen-tscmode.pod.7 b/docs/man/xen-tscmode.pod.7
index 3bbc96f201..122ae36679 100644
--- a/docs/man/xen-tscmode.pod.7
+++ b/docs/man/xen-tscmode.pod.7
@@ -99,6 +99,9 @@ whether or not the VM has been saved/restored/migrated
 
 =back
 
+If the tsc_mode is set to "default" the decision to emulate TSC can be
+tweaked further with the "vtsc_tolerance_khz" option.
+
 To understand this in more detail, the rest of this document must
 be read.
 
@@ -211,6 +214,19 @@ is emulated.  Note that, though emulated, the "apparent" 
TSC frequency
 will be the TSC frequency of the initial physical machine, even after
 migration.
 
+Since the calibration of the TSC frequency may not be 100% accurate, the
+exact value of the frequency can change even across reboots. This means
+also several otherwise identical systems can have a slightly different
+TSC frequency. As a result TSC access will be emulated if a domU is
+migrated from one host to another, identical host. To avoid the
+performance impact of TSC emulation a certain tolerance of the measured
+host TSC frequency can be specified with "vtsc_tolerance_khz". If the
+measured "cpu_khz" value is within the tolerance range, TSC access
+remains native. Otherwise it will be emulated. This allows to migrate
+domUs between identical hardware. If the domU will be migrated to a
+different kind of hardware, say from a "2.3GHz" to a "2.5GHz" system,
+TSC will be emualted to maintain the TSC frequency expected by the domU.
+
 For environments where both TSC-safeness AND highest performance
 even across migration is a requirement, application code can be specially
 modified to use an algorithm explicitly designed into Xen for this purpose.
diff --git a/docs/man/xl.cfg.pod.5.in b/docs/man/xl.cfg.pod.5.in
index 2c1a6e1422..aff16052ef 100644
--- a/docs/man/xl.cfg.pod.5.in
+++ b/docs/man/xl.cfg.pod.5.in
@@ -1891,6 +1891,16 @@ determined in a similar way to that of B TSC 
mode.
 
 Please see B for more information on this option.
 
+=item 

Re: [Xen-devel] [PATCH v1] fuzz: wrappers.c depends on x86_emulate.h

[Jan Beulich]

>>> On 29.03.18 at 14:01,  wrote:
> --- a/tools/fuzz/x86_instruction_emulator/Makefile
> +++ b/tools/fuzz/x86_instruction_emulator/Makefile
> @@ -18,6 +18,8 @@ asm:
>  
>  asm/%: asm ;
>  
> +wrappers.o: $(x86_emulate.h)
> +
>  x86-emulate.c x86-emulate.h wrappers.c: %:
>   [ -L $* ] || ln -sf $(XEN_ROOT)/tools/tests/x86_emulator/$*

It certainly feels odd to request a v2 on this simple a change, but
the addition shouldn't be put in a random place. Instead we already
have 

# x86-emulate.c will be implicit for both
x86-emulate.o x86-emulate-cov.o: x86_emulate/x86_emulate.c $(x86_emulate.h)

fuzz-emul.o fuzz-emulate-cov.o: $(x86_emulate.h)

which the new one should be grouped with. Otoh maybe I should
just move the line while committing.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v1] fuzz: wrappers.c depends on x86_emulate.h

[Olaf Hering]

In my automated SLE_11 builds I often see failures like that:

[   74s] wrappers.c:5:25: error: x86-emulate.h: No such file or directory
[   74s] make[6]: *** [wrappers.o] Error 1

Signed-off-by: Olaf Hering 
---
 tools/fuzz/x86_instruction_emulator/Makefile | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/fuzz/x86_instruction_emulator/Makefile 
b/tools/fuzz/x86_instruction_emulator/Makefile
index df04d09252..697bc5ea64 100644
--- a/tools/fuzz/x86_instruction_emulator/Makefile
+++ b/tools/fuzz/x86_instruction_emulator/Makefile
@@ -18,6 +18,8 @@ asm:
 
 asm/%: asm ;
 
+wrappers.o: $(x86_emulate.h)
+
 x86-emulate.c x86-emulate.h wrappers.c: %:
[ -L $* ] || ln -sf $(XEN_ROOT)/tools/tests/x86_emulator/$*
 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] seabios regression in staging

[Olaf Hering]

It seems the latest seabios that was pulled into staging recently fails
to compile, at least in SLE_11:

[   86s]   Compile checking out/src/hw/blockcmd.o
[   86s] src/hw/blockcmd.c: In function 'scsi_rep_luns_scan':
[   86s] src/hw/blockcmd.c:229: error: unknown field 'cdbcmd' specified in 
initializer
[   86s] src/hw/blockcmd.c:229: warning: missing braces around initializer
[   86s] src/hw/blockcmd.c:229: warning: (near initialization for 
'op.')
[   86s] src/hw/blockcmd.c:229: warning: initialization makes integer from 
pointer without a cast
[   86s] make[7]: *** [out/src/hw/blockcmd.o] Error 1

I can eventually trick the build to use gcc48 not only for xen but also
for tools, but I wonder if there is a way to fix this.

Upstream may have not fixed that yet:
https://github.com/coreboot/seabios/blame/master/src/block.h
https://github.com/coreboot/seabios/blame/master/src/hw/blockcmd.c

Olaf


signature.asc
Description: PGP signature
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] fuzz/wrappers.c fails to build due to missing x86-emulate.h

[Olaf Hering]

On Thu, Mar 29, Jan Beulich wrote:

> wrappers.o: $(x86_emulate.h)

Thanks. This did probably help, the build got further. Will send a patch.
But another unrelated regression appeared.

Olaf


signature.asc
Description: PGP signature
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Cut-off date for Xen 4.11 is March 30th, 2018

[Juergen Gross]

On 29/03/18 12:50, Wei Liu wrote:
> On Thu, Mar 29, 2018 at 04:35:27AM -0600, Jan Beulich wrote:
> On 29.03.18 at 12:22,  wrote:
>>> On 03/29/2018 10:56 AM, Juergen Gross wrote:
 On 29/03/18 11:53, George Dunlap wrote:
> On 03/29/2018 07:52 AM, Juergen Gross wrote:
>> Hi all,
>>
>> The cut-off date for Xen 4.11 is March 30th, 2018. If you want your
>> features to be included for the release, please make sure they are
>> committed by March 30th, 2018.
>
> March 30th is a public holiday here in the UK.  Is it the same in
> Germany?  Would it be OK to say that things sent on Friday can be
> committed on Tuesday 3 April if the appropriate maintainer wasn't around
> to review them?
>
> If not we should warn people to get their stuff reviewed today if at all
> possible.
>
> As it happens I'll be working Friday so I can check in stuff that's got
> the right Acks / R-b's; but I won't do last-pass reviews on behalf of
> maintainers.

 I already thought of shifting the freeze by one week. Main reason is
 that several maintainers seem to have a backlog of patches to review
 which IMO should make it into 4.11.

 Thoughts?
>>>
>>> Well there's a benefit to this, but also a risk: that people will begin
>>> to see the "hard freeze" as more like a "soft freeze", that will always
>>> be pushed back / flexed if you push hard enough.  Part of the purpose of
>>> setting the hard freeze months in advance is so that people can plan
>>> ahead and get stuff reviewed in time; part of the reason for having
>>> 6-month releases is so that the cost of delaying a feature / patchset to
>>> the next release isn't very high.
>>
>> As mentioned before I think anyway that we should revisit this
>> hard freeze date approach. I would much favor a hard freeze
>> date on where it is determined which features are intended to
>> make it and which not. Right now at least everything related to
>> Spectre and Meltdown would imo want to go into the category
>> of "we'll wait until it's in".
>>
> 
> You're mixing up two things: features and security fixes (and their
> subsequent patches). I agree the latter should get special attention
> because missing those would essentially render a release useless or
> unattractive.  Meltdown and Spectre fall into the second category, as
> with all the XSAs.

And we still have the possibility of individual Release-Acks.

> But most of the time, and most developers / contributors write new
> features.  If they are identified with strategic importance, we should
> wait (livepatching comes to mind), but for normal ones (which noone
> argues for), we should have the default position to not wait.
> 
> This isn't incompatible with what you said.

Right.

Still I think shifting by one week, given the current situation where
some maintainers had to spend a significant amount of the development
phase with security stuff instead of being able to review patches, is
a sensible thing to do.

So I think I'll do that with making it very clear that this won't be the
default process for the following releases.


Juergen

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v18 10/11] common: add a new mappable resource type: XENMEM_resource_grant_table

[Paul Durrant]

> -Original Message-
> From: Jan Beulich [mailto:jbeul...@suse.com]
> Sent: 26 March 2018 13:55
> To: Paul Durrant 
> Cc: Andrew Cooper ; Wei Liu
> ; George Dunlap ; Ian
> Jackson ; Stefano Stabellini
> ; xen-devel@lists.xenproject.org; Tim (Xen.org)
> 
> Subject: Re: [PATCH v18 10/11] common: add a new mappable resource
> type: XENMEM_resource_grant_table
> 
> >>> On 26.03.18 at 14:16,  wrote:
>  On 22.03.18 at 12:55,  wrote:
> >> --- a/xen/common/grant_table.c
> >> +++ b/xen/common/grant_table.c
> >> @@ -3863,6 +3863,35 @@ int mem_sharing_gref_to_gfn(struct
> grant_table *gt, grant_ref_t ref,
> >>  }
> >>  #endif
> >>
> >> +/* caller must hold read or write lock */
> >> +static int gnttab_get_status_frame_mfn(struct domain *d,
> >> +   unsigned long idx, mfn_t *mfn)
> >> +{
> >> +struct grant_table *gt = d->grant_table;
> >> +
> >> +if ( idx >= nr_status_frames(gt) )
> >> +return -EINVAL;
> >> +
> >> +*mfn = _mfn(virt_to_mfn(gt->status[idx]));
> >> +return 0;
> >> +}
> >> +
> >> +/* caller must hold write lock */
> >> +static int gnttab_get_shared_frame_mfn(struct domain *d,
> >> +   unsigned long idx, mfn_t *mfn)
> >> +{
> >> +struct grant_table *gt = d->grant_table;
> >> +
> >> +if ( (idx >= nr_grant_frames(gt)) && (idx < gt->max_grant_frames) )
> >> +gnttab_grow_table(d, idx + 1);
> >> +
> >> +if ( idx >= nr_grant_frames(gt) )
> >> +return -EINVAL;
> >> +
> >> +*mfn = _mfn(virt_to_mfn(gt->shared_raw[idx]));
> >> +return 0;
> >> +}
> >
> > I realize the anomaly was there already before, but imo it becomes
> > more pronounced with the two functions differing in more than just
> > the shared vs status naming (IOW I find it strange that one grows
> > the grant table while the other doesn't). This extends to ...
> >
> >> +int gnttab_get_shared_frame(struct domain *d, unsigned long idx,
> >> +mfn_t *mfn)
> >> +{
> >> +struct grant_table *gt = d->grant_table;
> >> +int rc;
> >> +
> >> +grant_write_lock(gt);
> >> +rc = gnttab_get_shared_frame_mfn(d, idx, mfn);
> >> +grant_write_unlock(gt);
> >> +
> >> +return rc;
> >> +}
> >> +
> >> +int gnttab_get_status_frame(struct domain *d, unsigned long idx,
> >> +mfn_t *mfn)
> >> +{
> >> +struct grant_table *gt = d->grant_table;
> >> +int rc;
> >> +
> >> +grant_read_lock(gt);
> >> +rc = gnttab_get_status_frame_mfn(d, idx, mfn);
> >> +grant_read_unlock(gt);
> >> +
> >> +return rc;
> >> +}
> >
> > ... these two acquiring the lock in different ways.

So, you want me to have gnttab_get_status_frame() grow the table accordingly? 
I'd really rather not do that at v19 of the series when it's never been part of 
the scope before.

> >
> > And then I'm completely missing the interaction with
> > gnttab_unpopulate_status_frames(). While this might not be a
> > practical problem at this point in time, we're liable to forget to
> > address this later on if there's no stop gap measure. A PV guest
> > mapping the obtained MFNs is going to be fine, but a HVM/PVH
> > one isn't, since neither x86 nor ARM refcount pages inserted into
> > or removed from a domain's p2m. I therefore think you need to
> > add a is_hvm_domain() check to acquire_grant_table(), with a
> > suitable fixme comment attached to it.
> 
> Or perhaps better paging_mode_translate(current->domain).
> 

Ok. I'll add the safety check and comment.

  Paul

> Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Cut-off date for Xen 4.11 is March 30th, 2018

[George Dunlap]

On 03/29/2018 11:35 AM, Jan Beulich wrote:
 On 29.03.18 at 12:22,  wrote:
>> On 03/29/2018 10:56 AM, Juergen Gross wrote:
>>> On 29/03/18 11:53, George Dunlap wrote:
 On 03/29/2018 07:52 AM, Juergen Gross wrote:
> Hi all,
>
> The cut-off date for Xen 4.11 is March 30th, 2018. If you want your
> features to be included for the release, please make sure they are
> committed by March 30th, 2018.

 March 30th is a public holiday here in the UK.  Is it the same in
 Germany?  Would it be OK to say that things sent on Friday can be
 committed on Tuesday 3 April if the appropriate maintainer wasn't around
 to review them?

 If not we should warn people to get their stuff reviewed today if at all
 possible.

 As it happens I'll be working Friday so I can check in stuff that's got
 the right Acks / R-b's; but I won't do last-pass reviews on behalf of
 maintainers.
>>>
>>> I already thought of shifting the freeze by one week. Main reason is
>>> that several maintainers seem to have a backlog of patches to review
>>> which IMO should make it into 4.11.
>>>
>>> Thoughts?
>>
>> Well there's a benefit to this, but also a risk: that people will begin
>> to see the "hard freeze" as more like a "soft freeze", that will always
>> be pushed back / flexed if you push hard enough.  Part of the purpose of
>> setting the hard freeze months in advance is so that people can plan
>> ahead and get stuff reviewed in time; part of the reason for having
>> 6-month releases is so that the cost of delaying a feature / patchset to
>> the next release isn't very high.
> 
> As mentioned before I think anyway that we should revisit this
> hard freeze date approach. I would much favor a hard freeze
> date on where it is determined which features are intended to
> make it and which not.
Well ultimately things take a non-deterministic amount of time; so we
can either choose "We must release by this date" and drop all features
not ready, or we can choose "We must release with this feature" and slip
until it's ready.

In general, people seem to prefer the former; and from an administrative
point of view it's certainly simpler than trying to determine which
feature will be blockers.

That said, it may make sense to argue that specific features / patchsets
should be blockers in exceptional circumstances.

> Right now at least everything related to
> Spectre and Meltdown would imo want to go into the category
> of "we'll wait until it's in".


As Wei said, bug fixes are always potential blockers (which is the point
of the freeze).  Are you thinking here specifically about the
performance improvements, or about some new functionality (which I
haven't noticed / heard of yet)?

If you're talking about the performance patches, how do we know when
we're "done"?  Are we going to accept only patches / improvements in the
current series, or are we going to allow people to add new techniques
they come up with until we can't think of any more?  Or until we reach a
specific performance level?

FWIW I do think getting the current XPTI performance improvement series
in for this release makes a lot of sense.

 -George

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 for-4.11] tools: set DEBUG_DIR from configure

[Wei Liu]

On Wed, Mar 28, 2018 at 08:34:14AM +0100, Roger Pau Monne wrote:
> Allow the path to be set from a configure command line option.
> 
> Signed-off-by: Roger Pau Monné 

Acked-by: Wei Liu 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Cut-off date for Xen 4.11 is March 30th, 2018

[Wei Liu]

On Thu, Mar 29, 2018 at 04:35:27AM -0600, Jan Beulich wrote:
> >>> On 29.03.18 at 12:22,  wrote:
> > On 03/29/2018 10:56 AM, Juergen Gross wrote:
> >> On 29/03/18 11:53, George Dunlap wrote:
> >>> On 03/29/2018 07:52 AM, Juergen Gross wrote:
>  Hi all,
> 
>  The cut-off date for Xen 4.11 is March 30th, 2018. If you want your
>  features to be included for the release, please make sure they are
>  committed by March 30th, 2018.
> >>>
> >>> March 30th is a public holiday here in the UK.  Is it the same in
> >>> Germany?  Would it be OK to say that things sent on Friday can be
> >>> committed on Tuesday 3 April if the appropriate maintainer wasn't around
> >>> to review them?
> >>>
> >>> If not we should warn people to get their stuff reviewed today if at all
> >>> possible.
> >>>
> >>> As it happens I'll be working Friday so I can check in stuff that's got
> >>> the right Acks / R-b's; but I won't do last-pass reviews on behalf of
> >>> maintainers.
> >> 
> >> I already thought of shifting the freeze by one week. Main reason is
> >> that several maintainers seem to have a backlog of patches to review
> >> which IMO should make it into 4.11.
> >> 
> >> Thoughts?
> > 
> > Well there's a benefit to this, but also a risk: that people will begin
> > to see the "hard freeze" as more like a "soft freeze", that will always
> > be pushed back / flexed if you push hard enough.  Part of the purpose of
> > setting the hard freeze months in advance is so that people can plan
> > ahead and get stuff reviewed in time; part of the reason for having
> > 6-month releases is so that the cost of delaying a feature / patchset to
> > the next release isn't very high.
> 
> As mentioned before I think anyway that we should revisit this
> hard freeze date approach. I would much favor a hard freeze
> date on where it is determined which features are intended to
> make it and which not. Right now at least everything related to
> Spectre and Meltdown would imo want to go into the category
> of "we'll wait until it's in".
> 

You're mixing up two things: features and security fixes (and their
subsequent patches). I agree the latter should get special attention
because missing those would essentially render a release useless or
unattractive.  Meltdown and Spectre fall into the second category, as
with all the XSAs.

But most of the time, and most developers / contributors write new
features.  If they are identified with strategic importance, we should
wait (livepatching comes to mind), but for normal ones (which noone
argues for), we should have the default position to not wait.

This isn't incompatible with what you said.

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v5] new config option vtsc_tolerance_khz to avoid TSC emulation

[Jan Beulich]

>>> On 29.03.18 at 12:05,  wrote:
> On Thu, Mar 29, Jan Beulich wrote:
> 
>> Actually I was wrong - we have an abstraction already, just that
>> it's upper case: ABS(). But it requires its input to have signed type.
> 
> Would this be acceptable?
> khz_diff = ABS((long)cpu_khz - (long)gtsc_khz);

In the worst case, yes. I'd prefer if you got away with just a single
cast though.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Cut-off date for Xen 4.11 is March 30th, 2018

[Jan Beulich]

>>> On 29.03.18 at 12:22,  wrote:
> On 03/29/2018 10:56 AM, Juergen Gross wrote:
>> On 29/03/18 11:53, George Dunlap wrote:
>>> On 03/29/2018 07:52 AM, Juergen Gross wrote:
 Hi all,

 The cut-off date for Xen 4.11 is March 30th, 2018. If you want your
 features to be included for the release, please make sure they are
 committed by March 30th, 2018.
>>>
>>> March 30th is a public holiday here in the UK.  Is it the same in
>>> Germany?  Would it be OK to say that things sent on Friday can be
>>> committed on Tuesday 3 April if the appropriate maintainer wasn't around
>>> to review them?
>>>
>>> If not we should warn people to get their stuff reviewed today if at all
>>> possible.
>>>
>>> As it happens I'll be working Friday so I can check in stuff that's got
>>> the right Acks / R-b's; but I won't do last-pass reviews on behalf of
>>> maintainers.
>> 
>> I already thought of shifting the freeze by one week. Main reason is
>> that several maintainers seem to have a backlog of patches to review
>> which IMO should make it into 4.11.
>> 
>> Thoughts?
> 
> Well there's a benefit to this, but also a risk: that people will begin
> to see the "hard freeze" as more like a "soft freeze", that will always
> be pushed back / flexed if you push hard enough.  Part of the purpose of
> setting the hard freeze months in advance is so that people can plan
> ahead and get stuff reviewed in time; part of the reason for having
> 6-month releases is so that the cost of delaying a feature / patchset to
> the next release isn't very high.

As mentioned before I think anyway that we should revisit this
hard freeze date approach. I would much favor a hard freeze
date on where it is determined which features are intended to
make it and which not. Right now at least everything related to
Spectre and Meltdown would imo want to go into the category
of "we'll wait until it's in".

Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v18 06/11] x86/hvm/ioreq: add a new mappable resource type...

[Paul Durrant]

> -Original Message-
> From: Jan Beulich [mailto:jbeul...@suse.com]
> Sent: 26 March 2018 12:55
> To: Paul Durrant 
> Cc: JulienGrall ; Andrew Cooper
> ; Wei Liu ; George
> Dunlap ; Ian Jackson ;
> Stefano Stabellini ; xen-devel@lists.xenproject.org;
> Konrad Rzeszutek Wilk ; Tim (Xen.org)
> 
> Subject: Re: [PATCH v18 06/11] x86/hvm/ioreq: add a new mappable
> resource type...
> 
> >>> On 22.03.18 at 12:55,  wrote:
> > ... XENMEM_resource_ioreq_server
> >
> > This patch adds support for a new resource type that can be mapped using
> > the XENMEM_acquire_resource memory op.
> >
> > If an emulator makes use of this resource type then, instead of mapping
> > gfns, the IOREQ server will allocate pages from the emulating domain's
> > heap. These pages will never be present in the P2M of the guest at any
> > point (and are not even shared with the guest) and so are not vulnerable to
> > any direct attack by the guest.
> 
> "allocate pages from the emulating domain's heap" is a sub-optimal
> (at least slightly misleading) description, due to your use of
> MEMF_no_refcount together with the fact that domain's don't
> really have their own heaps.
> 

Ok, I'll say 'allocate pages which are assigned to the emulating domain' 
instead.

> > +static int hvm_alloc_ioreq_mfn(struct hvm_ioreq_server *s, bool buf)
> > +{
> > +struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
> > +
> > +if ( iorp->page )
> > +{
> > +/*
> > + * If a guest frame has already been mapped (which may happen
> > + * on demand if hvm_get_ioreq_server_info() is called), then
> > + * allocating a page is not permitted.
> > + */
> > +if ( !gfn_eq(iorp->gfn, INVALID_GFN) )
> > +return -EPERM;
> > +
> > +return 0;
> > +}
> > +
> > +/*
> > + * Allocated IOREQ server pages are assigned to the emulating
> > + * domain, not the target domain. This is safe because the emulating
> > + * domain cannot be destroyed until the ioreq server is destroyed.
> > + * Also we must use MEMF_no_refcount otherwise page allocation
> > + * could fail if the emulating domain has already reached its
> > + * maximum allocation.
> > + */
> > +iorp->page = alloc_domheap_page(s->emulator, MEMF_no_refcount);
> > +
> > +if ( !iorp->page )
> > +return -ENOMEM;
> > +
> > +if ( !get_page_type(iorp->page, PGT_writable_page) )
> > +goto fail;
> > +
> > +iorp->va = __map_domain_page_global(iorp->page);
> > +if ( !iorp->va )
> > +goto fail;
> > +
> > +clear_page(iorp->va);
> > +return 0;
> > +
> > + fail:
> > +put_page_and_type(iorp->page);
> 
> This is wrong in case it's the get_page_type() which failed.
> 

Oh, I thought it was safe. I'll re-work the error path.

> > +int arch_acquire_resource(struct domain *d, unsigned int type,
> > +  unsigned int id, unsigned long frame,
> > +  unsigned int nr_frames, xen_pfn_t mfn_list[],
> > +  unsigned int *flags)
> > +{
> > +int rc;
> > +
> > +switch ( type )
> > +{
> > +case XENMEM_resource_ioreq_server:
> > +{
> > +ioservid_t ioservid = id;
> > +unsigned int i;
> > +
> > +rc = -EINVAL;
> > +if ( id != (unsigned int)ioservid )
> > +break;
> > +
> > +rc = 0;
> > +for ( i = 0; i < nr_frames; i++ )
> > +{
> > +mfn_t mfn;
> > +
> > +rc = hvm_get_ioreq_server_frame(d, id, frame + i, );
> > +if ( rc )
> > +break;
> > +
> > +mfn_list[i] = mfn_x(mfn);
> > +}
> > +
> > +/*
> > + * The frames will be assigned to the tools domain that created
> > + * the ioreq server.
> > + */
> 
> s/will be/have been/ and perhaps drop "tools"?
> 

Ok.

> > --- a/xen/include/asm-arm/mm.h
> > +++ b/xen/include/asm-arm/mm.h
> > @@ -374,6 +374,14 @@ static inline void put_page_and_type(struct
> page_info *page)
> >
> >  void clear_and_clean_page(struct page_info *page);
> >
> > +static inline int arch_acquire_resource(
> > +struct domain *d, unsigned int type, unsigned int id,
> > +unsigned long frame,unsigned int nr_frames, xen_pfn_t mfn_list[],
> 
> Missing blank.
> 

Ok.

  Paul

> Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Cut-off date for Xen 4.11 is March 30th, 2018

[George Dunlap]

On 03/29/2018 10:56 AM, Juergen Gross wrote:
> On 29/03/18 11:53, George Dunlap wrote:
>> On 03/29/2018 07:52 AM, Juergen Gross wrote:
>>> Hi all,
>>>
>>> The cut-off date for Xen 4.11 is March 30th, 2018. If you want your
>>> features to be included for the release, please make sure they are
>>> committed by March 30th, 2018.
>>
>> March 30th is a public holiday here in the UK.  Is it the same in
>> Germany?  Would it be OK to say that things sent on Friday can be
>> committed on Tuesday 3 April if the appropriate maintainer wasn't around
>> to review them?
>>
>> If not we should warn people to get their stuff reviewed today if at all
>> possible.
>>
>> As it happens I'll be working Friday so I can check in stuff that's got
>> the right Acks / R-b's; but I won't do last-pass reviews on behalf of
>> maintainers.
> 
> I already thought of shifting the freeze by one week. Main reason is
> that several maintainers seem to have a backlog of patches to review
> which IMO should make it into 4.11.
> 
> Thoughts?

Well there's a benefit to this, but also a risk: that people will begin
to see the "hard freeze" as more like a "soft freeze", that will always
be pushed back / flexed if you push hard enough.  Part of the purpose of
setting the hard freeze months in advance is so that people can plan
ahead and get stuff reviewed in time; part of the reason for having
6-month releases is so that the cost of delaying a feature / patchset to
the next release isn't very high.

"30 March is a public holiday in the maintainer's country, but I didn't
realize that because it's not a public holiday in mine" might be a
reasonable objection.  But "it just took longer than I / the maintainer
expected" is less reasonable.

But it's probably not a huge deal either way; you're the release
coordinator, so it's your call. :-)

 -George

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v5] new config option vtsc_tolerance_khz to avoid TSC emulation

[Olaf Hering]

On Thu, Mar 29, Jan Beulich wrote:

> Actually I was wrong - we have an abstraction already, just that
> it's upper case: ABS(). But it requires its input to have signed type.

Would this be acceptable?
khz_diff = ABS((long)cpu_khz - (long)gtsc_khz);

Olaf


signature.asc
Description: PGP signature
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] fuzz/wrappers.c fails to build due to missing x86-emulate.h

[Jan Beulich]

>>> On 29.03.18 at 11:46,  wrote:
> In my automated SLE_11 builds I often see failures like that:
> 
> [   74s] wrappers.c:5:25: error: x86-emulate.h: No such file or directory
> [   74s] make[6]: *** [wrappers.o] Error 1
> 
> Just retriggering the package build fixes the error. SLE11 has make-3.81.
> Is that version of make perhaps too old to recognize the dependencies?

No, there's

wrappers.o: $(x86_emulate.h)

missing as it looks. Mind sending a patch?

Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Cut-off date for Xen 4.11 is March 30th, 2018

[Jan Beulich]

>>> On 29.03.18 at 11:56,  wrote:
> On 29/03/18 11:53, George Dunlap wrote:
>> On 03/29/2018 07:52 AM, Juergen Gross wrote:
>>> Hi all,
>>>
>>> The cut-off date for Xen 4.11 is March 30th, 2018. If you want your
>>> features to be included for the release, please make sure they are
>>> committed by March 30th, 2018.
>> 
>> March 30th is a public holiday here in the UK.  Is it the same in
>> Germany?  Would it be OK to say that things sent on Friday can be
>> committed on Tuesday 3 April if the appropriate maintainer wasn't around
>> to review them?
>> 
>> If not we should warn people to get their stuff reviewed today if at all
>> possible.
>> 
>> As it happens I'll be working Friday so I can check in stuff that's got
>> the right Acks / R-b's; but I won't do last-pass reviews on behalf of
>> maintainers.
> 
> I already thought of shifting the freeze by one week. Main reason is
> that several maintainers seem to have a backlog of patches to review
> which IMO should make it into 4.11.
> 
> Thoughts?

+1 (albeit I won't be around myself to do reviews/commits next week)

Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Cut-off date for Xen 4.11 is March 30th, 2018

[Juergen Gross]

On 29/03/18 11:53, George Dunlap wrote:
> On 03/29/2018 07:52 AM, Juergen Gross wrote:
>> Hi all,
>>
>> The cut-off date for Xen 4.11 is March 30th, 2018. If you want your
>> features to be included for the release, please make sure they are
>> committed by March 30th, 2018.
> 
> March 30th is a public holiday here in the UK.  Is it the same in
> Germany?  Would it be OK to say that things sent on Friday can be
> committed on Tuesday 3 April if the appropriate maintainer wasn't around
> to review them?
> 
> If not we should warn people to get their stuff reviewed today if at all
> possible.
> 
> As it happens I'll be working Friday so I can check in stuff that's got
> the right Acks / R-b's; but I won't do last-pass reviews on behalf of
> maintainers.

I already thought of shifting the freeze by one week. Main reason is
that several maintainers seem to have a backlog of patches to review
which IMO should make it into 4.11.

Thoughts?


Juergen

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH RFC] x86/HVM: suppress I/O completion for port output

[Paul Durrant]

> -Original Message-
> From: Jan Beulich [mailto:jbeul...@suse.com]
> Sent: 29 March 2018 10:41
> To: Paul Durrant 
> Cc: Andrew Cooper ; xen-devel  de...@lists.xenproject.org>
> Subject: RE: [PATCH RFC] x86/HVM: suppress I/O completion for port output
> 
> >>> On 29.03.18 at 11:13,  wrote:
> >> From: Jan Beulich [mailto:jbeul...@suse.com]
> >> Sent: 29 March 2018 10:10
> >>
> >> >>> On 29.03.18 at 10:54,  wrote:
> >> >> --- a/xen/arch/x86/hvm/emulate.c
> >> >> +++ b/xen/arch/x86/hvm/emulate.c
> >> >> @@ -282,7 +282,7 @@ static int hvmemul_do_io(
> >> >>  rc = hvm_send_ioreq(s, , 0);
> >> >>  if ( rc != X86EMUL_RETRY || currd->is_shutting_down )
> >> >>  vio->io_req.state = STATE_IOREQ_NONE;
> >> >> -else if ( data_is_addr )
> >> >> +else if ( data_is_addr || (!is_mmio && dir == IOREQ_WRITE) 
> >> >> )
> >> >
> >> > I'm not entirely sure, but it seems like this test might actually be
> >> > !hvm_vcpu_io_need_completion()...
> >> >
> >> >>  rc = X86EMUL_OKAY;
> >> >>  }
> >> >>  break;
> >> >> --- a/xen/include/asm-x86/hvm/vcpu.h
> >> >> +++ b/xen/include/asm-x86/hvm/vcpu.h
> >> >> @@ -91,10 +91,12 @@ struct hvm_vcpu_io {
> >> >>  const struct g2m_ioport *g2m_ioport;
> >> >>  };
> >> >>
> >> >> -static inline bool_t hvm_vcpu_io_need_completion(const struct
> >> >> hvm_vcpu_io *vio)
> >> >> +static inline bool hvm_vcpu_io_need_completion(const struct
> >> >> hvm_vcpu_io *vio)
> >> >>  {
> >> >>  return (vio->io_req.state == STATE_IOREQ_READY) &&
> >> >> -   !vio->io_req.data_is_ptr;
> >> >> +   !vio->io_req.data_is_ptr &&
> >> >> +   (vio->io_req.type != IOREQ_TYPE_PIO ||
> >> >> +vio->io_req.dir != IOREQ_WRITE);
> >> >
> >> > ... now that you've updated it here.
> >>
> >> It could have been before, and it wasn't, so I didn't want to change
> >> that. My assumption is that the function wasn't used to leverage
> >> local variables (and avoid the .state comparison altogether).
> >
> > Yes, that's why it was like it is.
> >
> >> Technically it could be switched, I agree. I guess I should at least
> >> attach a comment, clarifying that this is an open-coded, slightly
> >> optimized variant of the function.
> >>
> >
> > Alternatively if the macro is modified to take an ioreq_t pointer directly
> > rather than a struct hvm_vcpu_io pointer, then I think you could just pass
> > the on-stack ioreq_t to it in hvmemul_do_io() and avoid any real need for
> the
> > open-coded test.
> 
> Hmm, yes, but even then I'm not sure the compiler would realize
> it can omit the .state check. I may try out that transformation once
> I know whether this helps in the first place.
> 

Ok. Fair enough.

  Paul

> Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v5] new config option vtsc_tolerance_khz to avoid TSC emulation

[Jan Beulich]

>>> On 29.03.18 at 11:23,  wrote:
> On Thu, Mar 29, Jan Beulich wrote:
> 
>> When you use abs() or alike in places like this, it is more immediately
>> obvious to the reader what you're doing.
> 
> Does every supported compiler actually understand this?
> int khz_diff = __builtin_abs(cpu_khz - gtsc_khz);
> Or do we need an inline abs() in case it is not gcc?

Actually I was wrong - we have an abstraction already, just that
it's upper case: ABS(). But it requires its input to have signed type.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v18 05/11] x86/mm: add HYPERVISOR_memory_op to acquire guest resources

[Paul Durrant]

> -Original Message-
> From: Jan Beulich [mailto:jbeul...@suse.com]
> Sent: 26 March 2018 12:41
> To: Paul Durrant 
> Cc: Julien Grall ; Andrew Cooper
> ; Wei Liu ; George
> Dunlap ; Ian Jackson ;
> Stefano Stabellini ; xen-devel@lists.xenproject.org;
> Konrad Rzeszutek Wilk ; Tim (Xen.org)
> 
> Subject: Re: [PATCH v18 05/11] x86/mm: add HYPERVISOR_memory_op to
> acquire guest resources
> 
> >>> On 22.03.18 at 12:55,  wrote:
> > --- a/xen/common/memory.c
> > +++ b/xen/common/memory.c
> > @@ -967,6 +967,94 @@ static long xatp_permission_check(struct domain
> *d, unsigned int space)
> >  return xsm_add_to_physmap(XSM_TARGET, current->domain, d);
> >  }
> >
> > +static int acquire_resource(
> > +XEN_GUEST_HANDLE_PARAM(xen_mem_acquire_resource_t) arg)
> > +{
> > +struct domain *d, *currd = current->domain;
> > +xen_mem_acquire_resource_t xmar;
> > +/*
> > + * The mfn_list and gfn_list (below) arrays are ok on stack for the
> > + * moment since they are small, but if they need to grow in future
> > + * use-cases then per-CPU arrays or heap allocations may be required.
> > + */
> > +xen_pfn_t mfn_list[2];
> > +int rc;
> > +
> > +if ( copy_from_guest(, arg, 1) )
> > +return -EFAULT;
> > +
> > +if ( xmar.flags != 0 )
> > +return -EINVAL;
> > +
> > +if ( guest_handle_is_null(xmar.frame_list) )
> > +{
> > +if ( xmar.nr_frames )
> > +return -EINVAL;
> > +
> > +xmar.nr_frames = ARRAY_SIZE(mfn_list);
> > +
> > +if ( __copy_field_to_guest(arg, , nr_frames) )
> > +return -EFAULT;
> > +
> > +return 0;
> > +}
> > +
> > +if ( xmar.nr_frames > ARRAY_SIZE(mfn_list) )
> > +return -E2BIG;
> > +
> > +rc = rcu_lock_remote_domain_by_id(xmar.domid, );
> > +if ( rc )
> > +return rc;
> > +
> > +rc = xsm_domain_resource_map(XSM_DM_PRIV, d);
> > +if ( rc )
> > +goto out;
> > +
> > +switch ( xmar.type )
> > +{
> > +default:
> > +rc = -EOPNOTSUPP;
> > +break;
> > +}
> > +
> > +if ( rc )
> > +goto out;
> > +
> > +if ( !paging_mode_translate(currd) )
> > +{
> > +if ( copy_to_guest(xmar.frame_list, mfn_list, xmar.nr_frames) )
> > +rc = -EFAULT;
> > +}
> > +else
> > +{
> > +xen_pfn_t gfn_list[ARRAY_SIZE(mfn_list)];
> > +unsigned int i;
> > +
> > +if ( copy_from_guest(gfn_list, xmar.frame_list, xmar.nr_frames) )
> > +rc = -EFAULT;
> > +
> > +for ( i = 0; !rc && i < xmar.nr_frames; i++ )
> > +{
> > +rc = set_foreign_p2m_entry(currd, gfn_list[i],
> > +   _mfn(mfn_list[i]));
> > +if ( rc )
> > +/*
> > + * Make sure rc is -EIO for any iteration other than
> > + * the first.
> > + */
> > +rc = i ? -EIO : rc;
> 
> Perhaps easier as
> 
> /*
>  * Make sure rc is -EIO for any iteration other than
>  * the first.
>  */
> if ( rc && i )
> rc = -EIO;
> 
> ? Looks like the comment could then also be a single line one.
> 

Ok.

> > +}
> > +}
> > +
> > +if ( xmar.flags != 0 &&
> > + __copy_field_to_guest(arg, , flags) )
> > +rc = -EFAULT;
> > +
> > + out:
> > +rcu_unlock_domain(d);
> > +return rc;
> > +}
> 
> Blank line please ahead of main "return".
> 

Ok.

> > --- a/xen/include/public/memory.h
> > +++ b/xen/include/public/memory.h
> > @@ -599,6 +599,59 @@ struct xen_reserved_device_memory_map {
> >  typedef struct xen_reserved_device_memory_map
> > xen_reserved_device_memory_map_t;
> >  DEFINE_XEN_GUEST_HANDLE(xen_reserved_device_memory_map_t);
> >
> > +/*
> > + * Get the pages for a particular guest resource, so that they can be
> > + * mapped directly by a tools domain.
> > + */
> > +#define XENMEM_acquire_resource 28
> > +struct xen_mem_acquire_resource {
> > +/* IN - The domain whose resource is to be mapped */
> > +domid_t domid;
> > +/* IN - the type of resource */
> > +uint16_t type;
> > +/*
> > + * IN - a type-specific resource identifier, which must be zero
> > + *  unless stated otherwise.
> > + */
> > +uint32_t id;
> > +/*
> > + * IN/OUT - As an IN parameter number of frames of the resource
> > + *  to be mapped. However, if the specified value is 0 and
> > + *  frame_list is NULL then this field will be set to the
> > + *  maximum value supported by the implementation on return.
> > + */
> > +uint32_t nr_frames;
> > +/*
> > + * OUT - Must be zero 

Re: [Xen-devel] Cut-off date for Xen 4.11 is March 30th, 2018

[George Dunlap]

On 03/29/2018 07:52 AM, Juergen Gross wrote:
> Hi all,
> 
> The cut-off date for Xen 4.11 is March 30th, 2018. If you want your
> features to be included for the release, please make sure they are
> committed by March 30th, 2018.

March 30th is a public holiday here in the UK.  Is it the same in
Germany?  Would it be OK to say that things sent on Friday can be
committed on Tuesday 3 April if the appropriate maintainer wasn't around
to review them?

If not we should warn people to get their stuff reviewed today if at all
possible.

As it happens I'll be working Friday so I can check in stuff that's got
the right Acks / R-b's; but I won't do last-pass reviews on behalf of
maintainers.

 -George

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [qemu-mainline test] 121325: regressions - FAIL

[osstest service owner]

flight 121325 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/121325/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-qemuu-nested-intel 14 xen-boot/l1   fail REGR. vs. 120095
 test-amd64-amd64-qemuu-nested-amd 14 xen-boot/l1 fail REGR. vs. 120095

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail  like 120095
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 120095
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 120095
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 120095
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 120095
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 120095
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-amd64-xl-pvhv2-intel 12 guest-start fail never pass
 test-amd64-amd64-xl-pvhv2-amd 12 guest-start  fail  never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass

version targeted for testing:
 qemuufa3704d87720d7049d483ff669b9e2ff991e7658
baseline version:
 qemuu6697439794f72b3501ee16bb95d16854f9981421

Last test of basis   120095  2018-02-28 13:46:33 Z   28 days
Failing since120146  2018-03-02 10:10:57 Z   26 days   16 attempts
Testing same since   121325  2018-03-28 09:25:38 Z0 days1 attempts


People who touched revisions under test:
  Alberto Garcia 
  Alex Bennée 
  Alex Bennée 
  Alex Williamson 
  Alexey Kardashevskiy 
  Alistair Francis 
  Alistair Francis 
  Andrew Jones 
  Andrey Smirnov 
  Anton 

[Xen-devel] fuzz/wrappers.c fails to build due to missing x86-emulate.h

[Olaf Hering]

In my automated SLE_11 builds I often see failures like that:

[   74s] wrappers.c:5:25: error: x86-emulate.h: No such file or directory
[   74s] make[6]: *** [wrappers.o] Error 1

Just retriggering the package build fixes the error. SLE11 has make-3.81.
Is that version of make perhaps too old to recognize the dependencies?

Olaf


signature.asc
Description: PGP signature
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH RFC] x86/HVM: suppress I/O completion for port output

[Jan Beulich]

>>> On 29.03.18 at 11:13,  wrote:
>> From: Jan Beulich [mailto:jbeul...@suse.com]
>> Sent: 29 March 2018 10:10
>> 
>> >>> On 29.03.18 at 10:54,  wrote:
>> >> --- a/xen/arch/x86/hvm/emulate.c
>> >> +++ b/xen/arch/x86/hvm/emulate.c
>> >> @@ -282,7 +282,7 @@ static int hvmemul_do_io(
>> >>  rc = hvm_send_ioreq(s, , 0);
>> >>  if ( rc != X86EMUL_RETRY || currd->is_shutting_down )
>> >>  vio->io_req.state = STATE_IOREQ_NONE;
>> >> -else if ( data_is_addr )
>> >> +else if ( data_is_addr || (!is_mmio && dir == IOREQ_WRITE) )
>> >
>> > I'm not entirely sure, but it seems like this test might actually be
>> > !hvm_vcpu_io_need_completion()...
>> >
>> >>  rc = X86EMUL_OKAY;
>> >>  }
>> >>  break;
>> >> --- a/xen/include/asm-x86/hvm/vcpu.h
>> >> +++ b/xen/include/asm-x86/hvm/vcpu.h
>> >> @@ -91,10 +91,12 @@ struct hvm_vcpu_io {
>> >>  const struct g2m_ioport *g2m_ioport;
>> >>  };
>> >>
>> >> -static inline bool_t hvm_vcpu_io_need_completion(const struct
>> >> hvm_vcpu_io *vio)
>> >> +static inline bool hvm_vcpu_io_need_completion(const struct
>> >> hvm_vcpu_io *vio)
>> >>  {
>> >>  return (vio->io_req.state == STATE_IOREQ_READY) &&
>> >> -   !vio->io_req.data_is_ptr;
>> >> +   !vio->io_req.data_is_ptr &&
>> >> +   (vio->io_req.type != IOREQ_TYPE_PIO ||
>> >> +vio->io_req.dir != IOREQ_WRITE);
>> >
>> > ... now that you've updated it here.
>> 
>> It could have been before, and it wasn't, so I didn't want to change
>> that. My assumption is that the function wasn't used to leverage
>> local variables (and avoid the .state comparison altogether).
> 
> Yes, that's why it was like it is.
> 
>> Technically it could be switched, I agree. I guess I should at least
>> attach a comment, clarifying that this is an open-coded, slightly
>> optimized variant of the function.
>> 
> 
> Alternatively if the macro is modified to take an ioreq_t pointer directly 
> rather than a struct hvm_vcpu_io pointer, then I think you could just pass 
> the on-stack ioreq_t to it in hvmemul_do_io() and avoid any real need for the 
> open-coded test.

Hmm, yes, but even then I'm not sure the compiler would realize
it can omit the .state check. I may try out that transformation once
I know whether this helps in the first place.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v18 05/11] x86/mm: add HYPERVISOR_memory_op to acquire guest resources

[Paul Durrant]

> -Original Message-
> From: Jan Beulich [mailto:jbeul...@suse.com]
> Sent: 26 March 2018 13:51
> To: Paul Durrant 
> Cc: Julien Grall ; Andrew Cooper
> ; Wei Liu ; George
> Dunlap ; Ian Jackson ;
> Stefano Stabellini ; xen-devel@lists.xenproject.org;
> Tim (Xen.org) 
> Subject: Re: [PATCH v18 05/11] x86/mm: add HYPERVISOR_memory_op to
> acquire guest resources
> 
> >>> On 26.03.18 at 13:41,  wrote:
>  On 22.03.18 at 12:55,  wrote:
> >> --- a/xen/include/public/memory.h
> >> +++ b/xen/include/public/memory.h
> >> @@ -599,6 +599,59 @@ struct xen_reserved_device_memory_map {
> >>  typedef struct xen_reserved_device_memory_map
> >> xen_reserved_device_memory_map_t;
> >>  DEFINE_XEN_GUEST_HANDLE(xen_reserved_device_memory_map_t);
> >>
> >> +/*
> >> + * Get the pages for a particular guest resource, so that they can be
> >> + * mapped directly by a tools domain.
> >> + */
> >> +#define XENMEM_acquire_resource 28
> >> +struct xen_mem_acquire_resource {
> >> +/* IN - The domain whose resource is to be mapped */
> >> +domid_t domid;
> >> +/* IN - the type of resource */
> >> +uint16_t type;
> >> +/*
> >> + * IN - a type-specific resource identifier, which must be zero
> >> + *  unless stated otherwise.
> >> + */
> >> +uint32_t id;
> >> +/*
> >> + * IN/OUT - As an IN parameter number of frames of the resource
> >> + *  to be mapped. However, if the specified value is 0 and
> >> + *  frame_list is NULL then this field will be set to the
> >> + *  maximum value supported by the implementation on return.
> >> + */
> >> +uint32_t nr_frames;
> >> +/*
> >> + * OUT - Must be zero on entry. On return this may contain a bitwise
> >> + *   OR of the following values.
> >> + */
> >> +uint32_t flags;
> >> +
> >> +/* The resource pages have been assigned to the tools domain */
> >> +#define _XENMEM_resource_flag_tools_owned 0
> >> +#define XENMEM_resource_flag_tools_owned (1u <<
> > _XENMEM_resource_flag_tools_owned)
> >
> > Is "tools" really an appropriate (and "flag" a necessary) name
> > component here? How about e.g. XENMEM_res_acq_caller_owned?
> 
> Or maybe XENMEM_rsrc_acq_caller_owned.
> 

Yes, I'm fine with that. I'll make the change.

  Paul

> Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 2/2] x86/setup: remap Xen image up to PFN_DOWN(__pa(_end))

[Daniel Kiper]

On Tue, Feb 13, 2018 at 02:16:22AM -0700, Jan Beulich wrote:
> >>> On 08.02.18 at 14:46,  wrote:
> > Sorry for late reply but I was busy with other stuff.
> >
> > On Fri, Jan 19, 2018 at 08:27:46AM -0700, Jan Beulich wrote:
> >> >>> On 10.01.18 at 14:05,  wrote:
> >> > Current limit, PFN_DOWN(xen_phys_start), introduced by commit b280442
> >> > (x86: make Xen early boot code relocatable) is not reliable. Potentially
> >> > its value may fall below PFN_DOWN(__pa(_end)) and then part of Xen image
> >> > may not be mapped after relocation. This will not happen in current code
> >> > thanks to "x86/setup: do not relocate over current Xen image placement"
> >> > patch. Though this safety measure may save a lot of debugging time when
> >> > somebody decide to relax existing relocation restrictions one day.
> >>
> >> I've gone back through the v2 discussion, and I continue to fail to
> >> see what is being fixed here, even if just theoretically. It is bad
> >
> > OK, let's give an example. I assume that there is no patch 1 and Xen can
> > relocate itself even it was initially relocated by the bootloader. So,
> > let's assume that the bootloader loaded Xen image at 0x8020
> > (xen_phys_start == 0x8000) and its size is 0x70 (7 MiB).
> > The RAM region ends at 0x80D0 and there is no RAM above that
> > address. At some point Xen realizes that it can relocate itself
> > to 0x8060 (xen_phys_start == 0x8040). So, it does and then
> > remaps itself. And here is the problem. Currently existing code
> > will remap only Xen image up to 0x803f. Everything above will
> > no be remapped. So, that is why I suggested this patch.
> >
> >> enough that the description here isn't clarifying this and I need to
> >> go back to the earlier discussion, but it's even worse if even that
> >> earlier discussion didn't really help. My conclusion is that you're
> >
> > Sorry about that.
> >
> >> talking about a case where old and positions of Xen overlap, a
> >> case which I thought patch 1 eliminates.
> >
> > It does not eliminate the issue described above. It just hides it.
>
> Well, no, I disagree - it makes an overlap impossible afaict,
> which is more that just hiding the problem. Anyway - I'm not
> going to object to the change provided it comes with a clear
> description of what _existing_ issue (even if just a theoretical
> one) is being fixed _with the currently present code in mind_
> (i.e. in particular including your patch 1).

Well, it looks that I have misread the code and I was simply lying.
Sorry about that. However, I had strange feeling that still something
is wrong here. And it is wrong. Currently destination region between
__image_base__ and (__image_base__ + XEN_IMG_OFFSET) may overlap with
the end of source image. And here is the problem. If anything between
__page_tables_start and __page_tables_end lands in the overlap then
some or even all page table entries may not be updated. This means boom
in early boot which will be difficult to the investigate. So, I think
the we have three choices to fix the issue:
  - drop XEN_IMG_OFFSET from
if ( (end > s) && (end - reloc_size + XEN_IMG_OFFSET >= __pa(_end)) )
  - add XEN_IMG_OFFSET to xen_phys_start in PFN_DOWN(xen_phys_start)
used in loops as one of conditions,
  - change PFN_DOWN(xen_phys_start) to PFN_DOWN(xen_remap_end_pfn)
proposed in this patch.

I think that we should choose first option. This way we will avoid
all kinds of overlaps which are always full can of worms.

Daniel

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v5 0/1] drm/xen-front: Add support for Xen PV display frontend

[Oleksandr Andrushchenko]

On 03/29/2018 12:22 PM, Oleksandr Andrushchenko wrote:

Changes since v4:

For your convenience I am attaching diff between v4..v5
diff --git a/Documentation/gpu/xen-front.rst b/Documentation/gpu/xen-front.rst
index 8188e03c9d23..009d942386c5 100644
--- a/Documentation/gpu/xen-front.rst
+++ b/Documentation/gpu/xen-front.rst
@@ -1,6 +1,6 @@
-
-Xen para-virtualized frontend driver
-
+
+ drm/xen-front Xen para-virtualized frontend driver
+
 
 This frontend driver implements Xen para-virtualized display
 according to the display protocol described at
diff --git a/drivers/gpu/drm/xen/xen_drm_front_evtchnl.c b/drivers/gpu/drm/xen/xen_drm_front_evtchnl.c
index e521785fd22b..02b6f3d9fe4c 100644
--- a/drivers/gpu/drm/xen/xen_drm_front_evtchnl.c
+++ b/drivers/gpu/drm/xen/xen_drm_front_evtchnl.c
@@ -186,8 +186,10 @@ static int evtchnl_alloc(struct xen_drm_front_info *front_info, int index,
 sring, XEN_PAGE_SIZE);
 
 		ret = xenbus_grant_ring(xb_dev, sring, 1, );
-		if (ret < 0)
+		if (ret < 0) {
+			free_page(page);
 			goto fail;
+		}
 
 		handler = evtchnl_interrupt_ctrl;
 	} else {
@@ -195,8 +197,10 @@ static int evtchnl_alloc(struct xen_drm_front_info *front_info, int index,
 
 		ret = gnttab_grant_foreign_access(xb_dev->otherend_id,
 virt_to_gfn((void *)page), 0);
-		if (ret < 0)
+		if (ret < 0) {
+			free_page(page);
 			goto fail;
+		}
 
 		gref = ret;
 		handler = evtchnl_interrupt_evt;
diff --git a/drivers/gpu/drm/xen/xen_drm_front_kms.c b/drivers/gpu/drm/xen/xen_drm_front_kms.c
index 545049dfaf0a..f3ef9dfb4dfb 100644
--- a/drivers/gpu/drm/xen/xen_drm_front_kms.c
+++ b/drivers/gpu/drm/xen/xen_drm_front_kms.c
@@ -107,12 +107,13 @@ static void send_pending_event(struct xen_drm_front_drm_pipeline *pipeline)
 }
 
 static void display_enable(struct drm_simple_display_pipe *pipe,
-		struct drm_crtc_state *crtc_state)
+		struct drm_crtc_state *crtc_state,
+		struct drm_plane_state *plane_state)
 {
 	struct xen_drm_front_drm_pipeline *pipeline =
 			to_xen_drm_pipeline(pipe);
 	struct drm_crtc *crtc = >crtc;
-	struct drm_framebuffer *fb = pipe->plane.state->fb;
+	struct drm_framebuffer *fb = plane_state->fb;
 	int ret, idx;
 
 	if (!drm_dev_enter(pipe->crtc.dev, ))
@@ -273,7 +274,7 @@ static void display_update(struct drm_simple_display_pipe *pipe,
 	drm_dev_exit(idx);
 }
 
-enum drm_mode_status display_mode_valid(struct drm_crtc *crtc,
+static enum drm_mode_status display_mode_valid(struct drm_crtc *crtc,
 		const struct drm_display_mode *mode)
 {
 	struct xen_drm_front_drm_pipeline *pipeline =
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v5] new config option vtsc_tolerance_khz to avoid TSC emulation

[Olaf Hering]

On Thu, Mar 29, Jan Beulich wrote:

> When you use abs() or alike in places like this, it is more immediately
> obvious to the reader what you're doing.

Does every supported compiler actually understand this?
int khz_diff = __builtin_abs(cpu_khz - gtsc_khz);
Or do we need an inline abs() in case it is not gcc?

Olaf


signature.asc
Description: PGP signature
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v5 0/1] drm/xen-front: Add support for Xen PV display frontend

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

Hello!

Boris/Daniel, I put your R-b tags, so please do let me know if this is not
acceptable, so I remove the tags.

This patch series adds support for Xen [1] para-virtualized
frontend display driver. It implements the protocol from
include/xen/interface/io/displif.h [2].
Accompanying backend [3] is implemented as a user-space application
and its helper library [4], capable of running as a Weston client
or DRM master.
Configuration of both backend and frontend is done via 
Xen guest domain configuration options [5].

***
* Driver limitations
***
 1. Configuration options 1.1 (contiguous display buffers) and 2 (backend
allocated buffers) below are not supported at the same time.

 2. Only primary plane without additional properties is supported.

 3. Only one video mode supported which resolution is configured via XenStore.

 4. All CRTCs operate at fixed frequency of 60Hz.

***
* Driver modes of operation in terms of display buffers used
***
 Depending on the requirements for the para-virtualized environment, namely
 requirements dictated by the accompanying DRM/(v)GPU drivers running in both
 host and guest environments, number of operating modes of para-virtualized
 display driver are supported:
  - display buffers can be allocated by either frontend driver or backend
  - display buffers can be allocated to be contiguous in memory or not

 Note! Frontend driver itself has no dependency on contiguous memory for
   its operation.

***
* 1. Buffers allocated by the frontend driver.
***

 The below modes of operation are configured at compile-time via
 frontend driver's kernel configuration.

 1.1. Front driver configured to use GEM CMA helpers
  This use-case is useful when used with accompanying DRM/vGPU driver in
  guest domain which was designed to only work with contiguous buffers,
  e.g. DRM driver based on GEM CMA helpers: such drivers can only import
  contiguous PRIME buffers, thus requiring frontend driver to provide
  such. In order to implement this mode of operation para-virtualized
  frontend driver can be configured to use GEM CMA helpers.

 1.2. Front driver doesn't use GEM CMA
  If accompanying drivers can cope with non-contiguous memory then, to
  lower pressure on CMA subsystem of the kernel, driver can allocate
  buffers from system memory.

 Note! If used with accompanying DRM/(v)GPU drivers this mode of operation
   may require IOMMU support on the platform, so accompanying DRM/vGPU
   hardware can still reach display buffer memory while importing PRIME
   buffers from the frontend driver.

***
* 2. Buffers allocated by the backend
***

 This mode of operation is run-time configured via guest domain configuration
 through XenStore entries.

 For systems which do not provide IOMMU support, but having specific
 requirements for display buffers it is possible to allocate such buffers
 at backend side and share those with the frontend.
 For example, if host domain is 1:1 mapped and has DRM/GPU hardware expecting
 physically contiguous memory, this allows implementing zero-copying
 use-cases.


I would like to thank at least, but not at last the following
people/communities who helped this driver to happen ;)

1. My team at EPAM for continuous support
2. Xen community for answering tons of questions on different
modes of operation of the driver with respect to virtualized
environment.
3. Rob Clark for "GEM allocation for para-virtualized DRM driver" [6]
4. Maarten Lankhorst for "Atomic driver and old remove FB behavior" [7]
5. Ville Syrjälä for "Questions on page flips and atomic modeset" [8]

Changes since v4:
***
- updated the driver after "drm/simple-kms-helper: Plumb plane state
  to the enable hook" [14]
- made display_mode_valid static
- fixed page leak on event channel error path
- changed title of the documentation to match the rest of the drivers
- removed from the series the patch from Noralf Trønnes [12] as it was sent out
  as a standalone one

Changes since v3:
***
- no changes to Xen related code (shared buffer handling, event channels etc.),
  but minor changes to xenbus_driver state machine due to re-worked unplug
  

[Xen-devel] [PATCH v5 1/1] drm/xen-front: Add support for Xen PV display frontend

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

Add support for Xen para-virtualized frontend display driver.
Accompanying backend [1] is implemented as a user-space application
and its helper library [2], capable of running as a Weston client
or DRM master.
Configuration of both backend and frontend is done via
Xen guest domain configuration options [3].

Driver limitations:
 1. Only primary plane without additional properties is supported.
 2. Only one video mode supported which resolution is configured via XenStore.
 3. All CRTCs operate at fixed frequency of 60Hz.

1. Implement Xen bus state machine for the frontend driver according to
the state diagram and recovery flow from display para-virtualized
protocol: xen/interface/io/displif.h.

2. Read configuration values from Xen store according
to xen/interface/io/displif.h protocol:
  - read connector(s) configuration
  - read buffer allocation mode (backend/frontend)

3. Handle Xen event channels:
  - create for all configured connectors and publish
corresponding ring references and event channels in Xen store,
so backend can connect
  - implement event channels interrupt handlers
  - create and destroy event channels with respect to Xen bus state

4. Implement shared buffer handling according to the
para-virtualized display device protocol at xen/interface/io/displif.h:
  - handle page directories according to displif protocol:
- allocate and share page directories
- grant references to the required set of pages for the
  page directory
  - allocate xen balllooned pages via Xen balloon driver
with alloc_xenballooned_pages/free_xenballooned_pages
  - grant references to the required set of pages for the
shared buffer itself
  - implement pages map/unmap for the buffers allocated by the
backend (gnttab_map_refs/gnttab_unmap_refs)

5. Implement kernel modesetiing/connector handling using
DRM simple KMS helper pipeline:

- implement KMS part of the driver with the help of DRM
  simple pipepline helper which is possible due to the fact
  that the para-virtualized driver only supports a single
  (primary) plane:
  - initialize connectors according to XenStore configuration
  - handle frame done events from the backend
  - create and destroy frame buffers and propagate those
to the backend
  - propagate set/reset mode configuration to the backend on display
enable/disable callbacks
  - send page flip request to the backend and implement logic for
reporting backend IO errors on prepare fb callback

- implement virtual connector handling:
  - support only pixel formats suitable for single plane modes
  - make sure the connector is always connected
  - support a single video mode as per para-virtualized driver
configuration

6. Implement GEM handling depending on driver mode of operation:
depending on the requirements for the para-virtualized environment, namely
requirements dictated by the accompanying DRM/(v)GPU drivers running in both
host and guest environments, number of operating modes of para-virtualized
display driver are supported:
 - display buffers can be allocated by either frontend driver or backend
 - display buffers can be allocated to be contiguous in memory or not

Note! Frontend driver itself has no dependency on contiguous memory for
its operation.

6.1. Buffers allocated by the frontend driver.

The below modes of operation are configured at compile-time via
frontend driver's kernel configuration.

6.1.1. Front driver configured to use GEM CMA helpers
 This use-case is useful when used with accompanying DRM/vGPU driver in
 guest domain which was designed to only work with contiguous buffers,
 e.g. DRM driver based on GEM CMA helpers: such drivers can only import
 contiguous PRIME buffers, thus requiring frontend driver to provide
 such. In order to implement this mode of operation para-virtualized
 frontend driver can be configured to use GEM CMA helpers.

6.1.2. Front driver doesn't use GEM CMA
 If accompanying drivers can cope with non-contiguous memory then, to
 lower pressure on CMA subsystem of the kernel, driver can allocate
 buffers from system memory.

Note! If used with accompanying DRM/(v)GPU drivers this mode of operation
may require IOMMU support on the platform, so accompanying DRM/vGPU
hardware can still reach display buffer memory while importing PRIME
buffers from the frontend driver.

6.2. Buffers allocated by the backend

This mode of operation is run-time configured via guest domain configuration
through XenStore entries.

For systems which do not provide IOMMU support, but having specific
requirements for display buffers it is possible to allocate such buffers
at backend side and share those with the frontend.
For example, if host domain is 1:1 mapped and has DRM/GPU hardware expecting
physically contiguous memory, this allows implementing zero-copying
use-cases.

Note, while using this scenario the following should be 

Re: [Xen-devel] [PATCH RFC] x86/HVM: suppress I/O completion for port output

[Paul Durrant]

> -Original Message-
> From: Jan Beulich [mailto:jbeul...@suse.com]
> Sent: 29 March 2018 10:10
> To: Paul Durrant 
> Cc: Andrew Cooper ; xen-devel  de...@lists.xenproject.org>
> Subject: RE: [PATCH RFC] x86/HVM: suppress I/O completion for port output
> 
> >>> On 29.03.18 at 10:54,  wrote:
> >> --- a/xen/arch/x86/hvm/emulate.c
> >> +++ b/xen/arch/x86/hvm/emulate.c
> >> @@ -282,7 +282,7 @@ static int hvmemul_do_io(
> >>  rc = hvm_send_ioreq(s, , 0);
> >>  if ( rc != X86EMUL_RETRY || currd->is_shutting_down )
> >>  vio->io_req.state = STATE_IOREQ_NONE;
> >> -else if ( data_is_addr )
> >> +else if ( data_is_addr || (!is_mmio && dir == IOREQ_WRITE) )
> >
> > I'm not entirely sure, but it seems like this test might actually be
> > !hvm_vcpu_io_need_completion()...
> >
> >>  rc = X86EMUL_OKAY;
> >>  }
> >>  break;
> >> --- a/xen/include/asm-x86/hvm/vcpu.h
> >> +++ b/xen/include/asm-x86/hvm/vcpu.h
> >> @@ -91,10 +91,12 @@ struct hvm_vcpu_io {
> >>  const struct g2m_ioport *g2m_ioport;
> >>  };
> >>
> >> -static inline bool_t hvm_vcpu_io_need_completion(const struct
> >> hvm_vcpu_io *vio)
> >> +static inline bool hvm_vcpu_io_need_completion(const struct
> >> hvm_vcpu_io *vio)
> >>  {
> >>  return (vio->io_req.state == STATE_IOREQ_READY) &&
> >> -   !vio->io_req.data_is_ptr;
> >> +   !vio->io_req.data_is_ptr &&
> >> +   (vio->io_req.type != IOREQ_TYPE_PIO ||
> >> +vio->io_req.dir != IOREQ_WRITE);
> >
> > ... now that you've updated it here.
> 
> It could have been before, and it wasn't, so I didn't want to change
> that. My assumption is that the function wasn't used to leverage
> local variables (and avoid the .state comparison altogether).

Yes, that's why it was like it is.

> Technically it could be switched, I agree. I guess I should at least
> attach a comment, clarifying that this is an open-coded, slightly
> optimized variant of the function.
> 

Alternatively if the macro is modified to take an ioreq_t pointer directly 
rather than a struct hvm_vcpu_io pointer, then I think you could just pass the 
on-stack ioreq_t to it in hvmemul_do_io() and avoid any real need for the 
open-coded test.

  Paul

> Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH RFC] x86/HVM: suppress I/O completion for port output

[Jan Beulich]

>>> On 29.03.18 at 10:54,  wrote:
>> --- a/xen/arch/x86/hvm/emulate.c
>> +++ b/xen/arch/x86/hvm/emulate.c
>> @@ -282,7 +282,7 @@ static int hvmemul_do_io(
>>  rc = hvm_send_ioreq(s, , 0);
>>  if ( rc != X86EMUL_RETRY || currd->is_shutting_down )
>>  vio->io_req.state = STATE_IOREQ_NONE;
>> -else if ( data_is_addr )
>> +else if ( data_is_addr || (!is_mmio && dir == IOREQ_WRITE) )
> 
> I'm not entirely sure, but it seems like this test might actually be 
> !hvm_vcpu_io_need_completion()...
> 
>>  rc = X86EMUL_OKAY;
>>  }
>>  break;
>> --- a/xen/include/asm-x86/hvm/vcpu.h
>> +++ b/xen/include/asm-x86/hvm/vcpu.h
>> @@ -91,10 +91,12 @@ struct hvm_vcpu_io {
>>  const struct g2m_ioport *g2m_ioport;
>>  };
>> 
>> -static inline bool_t hvm_vcpu_io_need_completion(const struct
>> hvm_vcpu_io *vio)
>> +static inline bool hvm_vcpu_io_need_completion(const struct
>> hvm_vcpu_io *vio)
>>  {
>>  return (vio->io_req.state == STATE_IOREQ_READY) &&
>> -   !vio->io_req.data_is_ptr;
>> +   !vio->io_req.data_is_ptr &&
>> +   (vio->io_req.type != IOREQ_TYPE_PIO ||
>> +vio->io_req.dir != IOREQ_WRITE);
> 
> ... now that you've updated it here.

It could have been before, and it wasn't, so I didn't want to change
that. My assumption is that the function wasn't used to leverage
local variables (and avoid the .state comparison altogether).
Technically it could be switched, I agree. I guess I should at least
attach a comment, clarifying that this is an open-coded, slightly
optimized variant of the function.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v5] new config option vtsc_tolerance_khz to avoid TSC emulation

[Roger Pau Monné]

On Thu, Mar 29, 2018 at 10:58:34AM +0200, Olaf Hering wrote:
> On Thu, Mar 29, Roger Pau Monné wrote:
> 
> > AFAICT in the chunk above you will disable vtsc without checking if
> > the hardware supports TSC scaling, which leads to inaccurate TSC values
> > on hardware that could provide accurate results without the software
> > emulation overhead.
> 
> Is that really the case? Maybe I get the logic wrong, but what I see is:
> what ever my change does, or if a HVM domain runs on a host with scaling
> feature, disable vtsc. hvm_get_tsc_scaling_ratio has no side effects.
> Isnt the purpose to not emulate vtsc if the hardware supports scaling?

Yes, that's correct. I read that wrong an somehow tied the vtsc
setting to the hardware TSC emulation.

IMO it would still be good to mention the relation between the
tolerance and the TSC hardware scaling in the commit message.

Thanks, Roger.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2] xen/acpi: off by one in read_acpi_id()

[Dan Carpenter]

If acpi_id is == nr_acpi_bits, then we access one element beyond the end
of the acpi_psd[] array or we set one bit beyond the end of the bit map
when we do __set_bit(acpi_id, acpi_id_present);

Fixes: 59a568029181 ("xen/acpi-processor: C and P-state driver that uploads 
said data to hypervisor.")
Signed-off-by: Dan Carpenter 
Reviewed-by: Joao Martins 
Reviewed-by: Juergen Gross 

diff --git a/drivers/xen/xen-acpi-processor.c b/drivers/xen/xen-acpi-processor.c
index c80195e8fbd1..b29f4e40851f 100644
--- a/drivers/xen/xen-acpi-processor.c
+++ b/drivers/xen/xen-acpi-processor.c
@@ -364,9 +364,9 @@ read_acpi_id(acpi_handle handle, u32 lvl, void *context, 
void **rv)
}
/* There are more ACPI Processor objects than in x2APIC or MADT.
 * This can happen with incorrect ACPI SSDT declerations. */
-   if (acpi_id > nr_acpi_bits) {
-   pr_debug("We only have %u, trying to set %u\n",
-nr_acpi_bits, acpi_id);
+   if (acpi_id >= nr_acpi_bits) {
+   pr_debug("max acpi id %u, trying to set %u\n",
+nr_acpi_bits - 1, acpi_id);
return AE_OK;
}
/* OK, There is a ACPI Processor object */

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] possible I/O emulation state machine issue

[Jan Beulich]

>>> On 29.03.18 at 10:42,  wrote:
>> From: Jan Beulich [mailto:jbeul...@suse.com]
>> Sent: 29 March 2018 07:27
>> 
>> Suppressing the stdvga port intercepts has, btw, not helped the
>> situation.
>> 
> 
> That surprises me. The whole string emulation should go out to QEMU without 
> being broken up in that case, and since it's an outsw I don't see why there 
> would be any retry of the linear->physical translation during completion.

See the patch sent earlier: HVMIO_mmio_completion means a full
second (or further) run through the emulator (which that patch
now avoids). Same would occur for an insn reading and writing
multiple memory locations, if at least the second one is in MMIO.
In that case we can't avoid the completion though, as the access
may additionally have been split (and we still need to execute
its later part(s)). To fully address this, I don't see a way around
recording completed steps (which is going to be a pretty intrusive
change as it looks).

Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] possible I/O emulation state machine issue

[Paul Durrant]

> -Original Message-
> From: Jan Beulich [mailto:jbeul...@suse.com]
> Sent: 29 March 2018 07:27
> To: Paul Durrant 
> Cc: Andrew Cooper ; xen-devel  de...@lists.xenproject.org>
> Subject: RE: possible I/O emulation state machine issue
> 
> >>> On 28.03.18 at 18:22,  wrote:
> >> From: Jan Beulich [mailto:jbeul...@suse.com]
> >> Sent: 28 March 2018 16:59
> >>
> >> Simply timing, perhaps. In any event, newest logs suggest we have
> >> an issue with Windows paging out the page the data for the
> >> REP OUTSW is coming from while the port I/O part of the operation
> >> is pending qemu's completion. Upon retry the linear->physical
> >> translation fails, and we leave incorrect state in place.
> >>
> >> I thought we cache the translation result, thus avoiding the need
> >> for a translation during the retry cycle, so either I'm misremembering
> >> or this doesn't work as intended. And in fact doing the translation a
> >> second time (with the potential of it failing) is wrong here - when the
> >> port access has occurred, we must not fail the emulation anymore
> >> (repeating the port write would probably be fine for the VGA, but
> >> would hardly be fine for e.g. an IDE interface).
> >
> > Yes, I thought we made sure all reps were completed using cached
> > translations before returning to guest.
> 
> We do this only for actual MMIO accesses, not for RAM ones,
> afaics.
> 
> I think I see a way to deal with the specific case here, but we'll
> certainly need to make things work properly in the general case.
> That's not something reasonable to be done for 4.11 though.
> 

Page table modification racing with an emulation sounds pretty bad though. I 
guess that if the damage is only limited to the guest though it's not something 
that requires immediate fix.

> Suppressing the stdvga port intercepts has, btw, not helped the
> situation.
> 

That surprises me. The whole string emulation should go out to QEMU without 
being broken up in that case, and since it's an outsw I don't see why there 
would be any retry of the linear->physical translation during completion.

  Paul

> Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v5] new config option vtsc_tolerance_khz to avoid TSC emulation

[Jan Beulich]

>>> On 29.03.18 at 10:17,  wrote:
> On Thu, Mar 29, Jan Beulich wrote:
> 
>> >>> On 27.03.18 at 11:26,  wrote:
>> > +khz_diff = cpu_khz > gtsc_khz ?
>> > +   cpu_khz - gtsc_khz : gtsc_khz - cpu_khz;
>> abs() (or some variant of it, like __builtin_absl(), seeing that we
>> don't appear to have any abstraction right now)?
> 
> I see no other usage of *abs*. Really optimize that one-shot function?

When you use abs() or alike in places like this, it is more immediately
obvious to the reader what you're doing.

>> d%d
> 
> A for an unsigned type? But I see it is done elsewhere, so it must be
> correct.

The type seen by printk() is "int" (i.e. signed) due to the way C's
integral type promotions work.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel