Re: [Xen-devel] [PATCH v6 11/12] xen: make grant resource limits per domain

[Juergen Gross]

On 13/09/17 17:46, Juergen Gross wrote:
> Instead of using the same global resource limits of grant tables (max.
> number of grant frames, max. number of maptrack frames) for all domains
> make these limits per domain. This will allow setting individual limits
> in the future. For now initialize the per domain limits with the global
> values.

Wrong commit message again.


Juergen

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v6 09/12] xen: delay allocation of grant table sub structures

[Juergen Gross]

On 13/09/17 17:46, Juergen Gross wrote:
> Delay the allocation of the grant table sub structures in order to
> allow modifying parameters needed for sizing of these structures at a
> per domain basis. Either do it from gnttab_grow_table() or just
> before the domain is started the first time.

Uuh, sorry, I forgot to update the commit message.


Juergen

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] implement runqueue per cpupool

[Juergen Gross]

On 12/09/17 02:45, anshulmakkar wrote:
> Attached patch series introduces the concept of  runqueue
> per cpupool.
> 
> "runqueue" configuration option can be specified with xl command
> while configuring cpupool. This will define the basis for 
> grouping of cpus (cpu, core, socket, all) in that cpupool.
> 
> Series is combined of following patches:
> [PATCH 1/3]: libxc related changes to add support for runqueue per cpupool
> [PATCH 2/3]: libxl related changes to add support for runqueue per cpupool
> [PATCH 3/3]: xen related changes.
> 
> On similar lines we can also have runqueue per pool configuration parameter
> for Credit scheduler. I plan to send a separate patch series for credit
> specific implementation.
> 

This series won't compile when patch 3 isn't applied.

You need to split patch 3 into two: one patch to extend the hypercall
interface which should go in first, and the other part of the hypervisor
code being the last patch.


Juergen

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [qemu-mainline test] 113391: regressions - FAIL

[osstest service owner]

flight 113391 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/113391/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemuu-win7-amd64 18 guest-start/win.repeat fail REGR. vs. 
113302

Regressions which are regarded as allowable (not blocking):
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stopfail REGR. vs. 113302

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 113302
 test-armhf-armhf-xl-rtds 16 guest-start/debian.repeatfail  like 113302
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 113302
 test-amd64-amd64-xl-rtds 10 debian-install   fail  like 113302
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail  like 113302
 test-amd64-amd64-xl-qemuu-ws16-amd64 10 windows-installfail never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qcow2 12 migrate-support-checkfail  never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 13 guest-saverestore   fail never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass

version targeted for testing:
 qemuu04ef33052c205170c92df21ca0b4be4f3b102188
baseline version:
 qemuua6e8c1dacfd37d34542e33600dcc50b7683b735a

Last test of basis   113302  2017-09-11 10:18:16 Z2 days
Testing same since   113345  2017-09-12 00:21:07 Z2 days3 attempts


People who touched revisions under test:
  Mark Cave-Ayland 
  Peter Maydell 
  Philippe Mathieu-Daudé 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl  pass
 test-armhf-armhf-xl

[Xen-devel] [xen-unstable-smoke test] 113429: regressions - FAIL

[osstest service owner]

flight 113429 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/113429/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemuu-debianhvm-i386 15 guest-saverestore.2 fail REGR. vs. 
113384

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  ff7eaa55e5055f65f0af8e18398e600f910bcdb1
baseline version:
 xen  082fc63f20e827eb0229d520b4ebf54140d9b21b

Last test of basis   113384  2017-09-12 23:14:17 Z1 days
Failing since113403  2017-09-13 09:03:32 Z0 days7 attempts
Testing same since   113417  2017-09-13 17:17:25 Z0 days4 attempts


People who touched revisions under test:
  Andrew Cooper 
  Haozhong Zhang 
  Jan Beulich 
  Oleksandr Grytsov 
  Petre Pircalabu 
  Wei Liu 

jobs:
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-amd64-amd64-xl-qemuu-debianhvm-i386 fail
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 341 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 0/6] remove xen dom0 support in DPDK

[Jianfeng Tan]

v2:
  - Address Bruce's comment on testpmd's memory init for xenvirt in patch 2.
  - Update rte_eal_version.map in bsd and eal.
  - Switch patch 5 and patch 6 so that we bump library version just once.

Following the calls on the mailing list:
http://dpdk.org/ml/archives/dev/2017-June/068151.html
The Technical Board decided to drop Xen dom0 support from EAL:
http://dpdk.org/ml/archives/dev/2017-June/068615.html

This series remove xen dom0 support in DPDK, as well as xenvirt PMD and
vhost_xen example.

What are effected?

After these patches, users cannot run DPDK applications inside xen dom0.

What are not effected?

Users can still run DPDK applications inside xen domU on pass-throughed
physical devices and virtio devices; on the host, users still can run
DPDK applications same as before.

Jianfeng Tan (6):
  examples/vhost_xen: remove
  net/xenvirt: remove
  xen: remove xen dependency in app, examples, test
  xen: remove xen dependency in drivers, ether, mempool
  eal: remove API rte_mem_phy2mch
  eal: remove xen dom0 support

 MAINTAINERS|   10 -
 app/test-pmd/Makefile  |4 -
 app/test-pmd/testpmd.c |   51 +-
 config/common_base |   10 -
 config/defconfig_arm-armv7a-linuxapp-gcc   |1 -
 doc/guides/contributing/documentation.rst  |1 -
 doc/guides/index.rst   |1 -
 doc/guides/linux_gsg/build_sample_apps.rst |5 +-
 doc/guides/linux_gsg/sys_reqs.rst  |   53 -
 doc/guides/nics/features/xenvirt.ini   |6 -
 doc/guides/prog_guide/env_abstraction_layer.rst|   11 -
 doc/guides/prog_guide/source_org.rst   |1 -
 doc/guides/rel_notes/deprecation.rst   |3 -
 doc/guides/rel_notes/release_17_11.rst |   14 +
 doc/guides/testpmd_app_ug/run_app.rst  |4 -
 doc/guides/xen/img/dpdk_xen_pkt_switch.png |  Bin 163842 -> 0 bytes
 doc/guides/xen/img/grant_refs.png  |  Bin 6405 -> 0 bytes
 doc/guides/xen/img/grant_table.png |  Bin 96762 -> 0 bytes
 doc/guides/xen/index.rst   |   38 -
 doc/guides/xen/pkt_switch.rst  |  470 --
 drivers/crypto/qat/qat_qp.c|7 +-
 drivers/net/Makefile   |2 -
 drivers/net/e1000/em_rxtx.c|4 +-
 drivers/net/e1000/igb_rxtx.c   |4 +-
 drivers/net/fm10k/fm10k_ethdev.c   |4 +-
 drivers/net/i40e/i40e_ethdev.c |2 +-
 drivers/net/i40e/i40e_fdir.c   |2 +-
 drivers/net/i40e/i40e_rxtx.c   |   16 +-
 drivers/net/ixgbe/ixgbe_rxtx.c |4 +-
 drivers/net/sfc/sfc.c  |2 +-
 drivers/net/xenvirt/Makefile   |   57 -
 drivers/net/xenvirt/rte_eth_xenvirt.c  |  766 --
 drivers/net/xenvirt/rte_eth_xenvirt.h  |   61 -
 drivers/net/xenvirt/rte_eth_xenvirt_version.map|7 -
 drivers/net/xenvirt/rte_mempool_gntalloc.c |  295 
 drivers/net/xenvirt/rte_xen_lib.c  |  454 --
 drivers/net/xenvirt/rte_xen_lib.h  |  116 --
 drivers/net/xenvirt/virtio_logs.h  |   70 -
 drivers/net/xenvirt/virtqueue.h|  273 
 examples/Makefile  |1 -
 examples/ip_pipeline/app.h |4 -
 examples/ip_pipeline/config_parse.c|   19 -
 examples/ip_pipeline/init.c|5 -
 examples/kni/main.c|3 -
 examples/vhost_xen/Makefile|   52 -
 examples/vhost_xen/main.c  | 1522 
 examples/vhost_xen/main.h  |   66 -
 examples/vhost_xen/vhost_monitor.c |  595 
 examples/vhost_xen/virtio-net.h|  113 --
 examples/vhost_xen/xen_vhost.h |  148 --
 examples/vhost_xen/xenstore_parse.c|  775 --
 lib/librte_eal/bsdapp/eal/Makefile |2 +-
 .../bsdapp/eal/include/exec-env/rte_dom0_common.h  |  107 --
 lib/librte_eal/bsdapp/eal/rte_eal_version.map  |3 -
 lib/librte_eal/common/eal_common_options.c |3 -
 lib/librte_eal/common/eal_internal_cfg.h   |1 -
 lib/librte_eal/common/eal_options.h|2 -
 lib/librte_eal/common/include/rte_memory.h |   71 -
 lib/librte_eal/linuxapp/Makefile   |2 -
 lib/librte_eal/linuxapp/eal/Makefile   |7 +-
 lib/librte_eal/linuxapp/eal/eal.c  |   24 -
 lib/librte_eal/linuxapp/eal/eal_memory.c   |   56 -
 

[Xen-devel] [PATCH v2 1/6] examples/vhost_xen: remove

[Jianfeng Tan]

Signed-off-by: Jianfeng Tan 
Acked-by: Bruce Richardson 
---
 MAINTAINERS |1 -
 examples/Makefile   |1 -
 examples/vhost_xen/Makefile |   52 --
 examples/vhost_xen/main.c   | 1522 ---
 examples/vhost_xen/main.h   |   66 --
 examples/vhost_xen/vhost_monitor.c  |  595 --
 examples/vhost_xen/virtio-net.h |  113 ---
 examples/vhost_xen/xen_vhost.h  |  148 
 examples/vhost_xen/xenstore_parse.c |  775 --
 9 files changed, 3273 deletions(-)
 delete mode 100644 examples/vhost_xen/Makefile
 delete mode 100644 examples/vhost_xen/main.c
 delete mode 100644 examples/vhost_xen/main.h
 delete mode 100644 examples/vhost_xen/vhost_monitor.c
 delete mode 100644 examples/vhost_xen/virtio-net.h
 delete mode 100644 examples/vhost_xen/xen_vhost.h
 delete mode 100644 examples/vhost_xen/xenstore_parse.c

diff --git a/MAINTAINERS b/MAINTAINERS
index a0cd75e..fe6c6db 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -196,7 +196,6 @@ F: lib/librte_eal/linuxapp/eal/*xen*
 F: lib/librte_eal/linuxapp/eal/include/exec-env/rte_dom0_common.h
 F: drivers/net/xenvirt/
 F: doc/guides/xen/
-F: examples/vhost_xen/
 F: doc/guides/nics/features/xenvirt.ini
 
 FreeBSD EAL (with overlaps)
diff --git a/examples/Makefile b/examples/Makefile
index 28354ff..d27eddd 100644
--- a/examples/Makefile
+++ b/examples/Makefile
@@ -89,7 +89,6 @@ DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination
 endif
 DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer
 DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi
-DIRS-$(CONFIG_RTE_LIBRTE_XEN_DOM0) += vhost_xen
 DIRS-y += vmdq
 DIRS-y += vmdq_dcb
 ifeq ($(CONFIG_RTE_LIBRTE_POWER), y)
diff --git a/examples/vhost_xen/Makefile b/examples/vhost_xen/Makefile
deleted file mode 100644
index ad2466a..000
--- a/examples/vhost_xen/Makefile
+++ /dev/null
@@ -1,52 +0,0 @@
-#   BSD LICENSE
-#
-#   Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
-#   All rights reserved.
-#
-#   Redistribution and use in source and binary forms, with or without
-#   modification, are permitted provided that the following conditions
-#   are met:
-#
-# * Redistributions of source code must retain the above copyright
-#   notice, this list of conditions and the following disclaimer.
-# * Redistributions in binary form must reproduce the above copyright
-#   notice, this list of conditions and the following disclaimer in
-#   the documentation and/or other materials provided with the
-#   distribution.
-# * Neither the name of Intel Corporation nor the names of its
-#   contributors may be used to endorse or promote products derived
-#   from this software without specific prior written permission.
-#
-#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-ifeq ($(RTE_SDK),)
-$(error "Please define RTE_SDK environment variable")
-endif
-
-# Default target, can be overridden by command line or environment
-RTE_TARGET ?= x86_64-native-linuxapp-gcc
-
-include $(RTE_SDK)/mk/rte.vars.mk
-
-# binary name
-APP = vhost-switch
-
-# all source are stored in SRCS-y
-SRCS-y := main.c vhost_monitor.c xenstore_parse.c
-
-CFLAGS += -O2 -I/usr/local/include -D_FILE_OFFSET_BITS=64 -Wno-unused-parameter
-CFLAGS += $(WERROR_FLAGS)
-CFLAGS += -D_GNU_SOURCE
-LDFLAGS += -lxenstore
-
-include $(RTE_SDK)/mk/rte.extapp.mk
diff --git a/examples/vhost_xen/main.c b/examples/vhost_xen/main.c
deleted file mode 100644
index eba4d35..000
--- a/examples/vhost_xen/main.c
+++ /dev/null
@@ -1,1522 +0,0 @@
-/*-
- *   BSD LICENSE
- *
- *   Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
- *   All rights reserved.
- *
- *   Redistribution and use in source and binary forms, with or without
- *   modification, are permitted provided that the following conditions
- *   are met:
- *
- * * Redistributions of source code must retain the above copyright
- *   notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- *   notice, this list of conditions and the following disclaimer in
- *   the 

[Xen-devel] [PATCH v2 3/6] xen: remove xen dependency in app, examples, test

[Jianfeng Tan]

Signed-off-by: Jianfeng Tan 
Acked-by: Bruce Richardson 
---
 examples/ip_pipeline/app.h  |  4 --
 examples/ip_pipeline/config_parse.c | 19 -
 examples/ip_pipeline/init.c |  5 ---
 examples/kni/main.c |  3 --
 test/test/process.h | 10 -
 test/test/test.c|  4 --
 test/test/test_eal_flags.c  | 81 -
 7 files changed, 126 deletions(-)

diff --git a/examples/ip_pipeline/app.h b/examples/ip_pipeline/app.h
index e41290e..94e7a6d 100644
--- a/examples/ip_pipeline/app.h
+++ b/examples/ip_pipeline/app.h
@@ -428,10 +428,6 @@ struct app_eal_params {
/* Interrupt mode for VFIO (legacy|msi|msix) */
char *vfio_intr;
 
-   /* Support running on Xen dom0 without hugetlbfs */
-   uint32_t xen_dom0_present;
-   int xen_dom0;
-
uint32_t parsed;
 };
 
diff --git a/examples/ip_pipeline/config_parse.c 
b/examples/ip_pipeline/config_parse.c
index 0b76134..3211c6a 100644
--- a/examples/ip_pipeline/config_parse.c
+++ b/examples/ip_pipeline/config_parse.c
@@ -809,21 +809,6 @@ parse_eal(struct app_params *app,
continue;
}
 
-   /* xen_dom0 */
-   if (strcmp(entry->name, "xen_dom0") == 0) {
-   int val;
-
-   PARSE_ERROR_DUPLICATE((p->xen_dom0_present == 0),
-   section_name,
-   entry->name);
-   p->xen_dom0_present = 1;
-
-   val = parser_read_arg_bool(entry->value);
-   PARSE_ERROR((val >= 0), section_name, entry->name);
-   p->xen_dom0 = val;
-   continue;
-   }
-
/* unrecognized */
PARSE_ERROR_INVALID(0, section_name, entry->name);
}
@@ -2643,10 +2628,6 @@ save_eal_params(struct app_params *app, FILE *f)
if (p->vfio_intr)
fprintf(f, "%s = %s\n", "vfio_intr", p->vfio_intr);
 
-   if (p->xen_dom0_present)
-   fprintf(f, "%s = %s\n", "xen_dom0",
-   (p->xen_dom0) ? "yes" : "no");
-
fputc('\n', f);
 }
 
diff --git a/examples/ip_pipeline/init.c b/examples/ip_pipeline/init.c
index 7cde49a..034c238 100644
--- a/examples/ip_pipeline/init.c
+++ b/examples/ip_pipeline/init.c
@@ -296,11 +296,6 @@ app_init_eal(struct app_params *app)
app->eal_argv[n_args++] = strdup(buffer);
}
 
-   if ((p->xen_dom0_present) && (p->xen_dom0)) {
-   snprintf(buffer, sizeof(buffer), "--xen-dom0");
-   app->eal_argv[n_args++] = strdup(buffer);
-   }
-
snprintf(buffer, sizeof(buffer), "--");
app->eal_argv[n_args++] = strdup(buffer);
 
diff --git a/examples/kni/main.c b/examples/kni/main.c
index e3bc2fb..9f9d227 100644
--- a/examples/kni/main.c
+++ b/examples/kni/main.c
@@ -919,9 +919,6 @@ main(int argc, char** argv)
continue;
kni_free_kni(port);
}
-#ifdef RTE_LIBRTE_XEN_DOM0
-   rte_kni_close();
-#endif
for (i = 0; i < RTE_MAX_ETHPORTS; i++)
if (kni_port_params_array[i]) {
rte_free(kni_port_params_array[i]);
diff --git a/test/test/process.h b/test/test/process.h
index 4f8d121..51ced12 100644
--- a/test/test/process.h
+++ b/test/test/process.h
@@ -52,11 +52,7 @@ static inline int
 process_dup(const char *const argv[], int numargs, const char *env_value)
 {
int num;
-#ifdef RTE_LIBRTE_XEN_DOM0
-   char *argv_cpy[numargs + 2];
-#else
char *argv_cpy[numargs + 1];
-#endif
int i, fd, status;
char path[32];
 
@@ -67,14 +63,8 @@ process_dup(const char *const argv[], int numargs, const 
char *env_value)
/* make a copy of the arguments to be passed to exec */
for (i = 0; i < numargs; i++)
argv_cpy[i] = strdup(argv[i]);
-#ifdef RTE_LIBRTE_XEN_DOM0
-   argv_cpy[i] = strdup("--xen-dom0");
-   argv_cpy[i + 1] = NULL;
-   num = numargs + 1;
-#else
argv_cpy[i] = NULL;
num = numargs;
-#endif
 
/* close all open file descriptors, check /proc/self/fd to only
 * call close on open fds. Exclude fds 0, 1 and 2*/
diff --git a/test/test/test.c b/test/test/test.c
index c561eb5..9accbd1 100644
--- a/test/test/test.c
+++ b/test/test/test.c
@@ -87,11 +87,7 @@ do_recursive_call(void)
{ "test_invalid_b_flag", no_action },
{ "test_invalid_vdev_flag", no_action },
{ "test_invalid_r_flag", no_action },
-#ifdef RTE_LIBRTE_XEN_DOM0
-   { "test_dom0_misc_flags", no_action },
-#else
{ "test_misc_flags", no_action },
-#endif
   

[Xen-devel] [PATCH v2 4/6] xen: remove xen dependency in drivers, ether, mempool

[Jianfeng Tan]

Signed-off-by: Jianfeng Tan 
Acked-by: Bruce Richardson 
---
 drivers/crypto/qat/qat_qp.c  | 7 +--
 drivers/net/i40e/i40e_rxtx.c | 8 ++--
 lib/librte_ether/rte_ethdev.c| 7 +--
 lib/librte_mempool/rte_mempool.c | 8 ++--
 4 files changed, 6 insertions(+), 24 deletions(-)

diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 5048d21..34f75ca 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -122,14 +122,9 @@ queue_dma_zone_reserve(const char *queue_name, uint32_t 
queue_size,
break;
default:
memzone_flags = RTE_MEMZONE_SIZE_HINT_ONLY;
-}
-#ifdef RTE_LIBRTE_XEN_DOM0
-   return rte_memzone_reserve_bounded(queue_name, queue_size,
-   socket_id, 0, RTE_CACHE_LINE_SIZE, RTE_PGSIZE_2M);
-#else
+   }
return rte_memzone_reserve_aligned(queue_name, queue_size, socket_id,
memzone_flags, queue_size);
-#endif
 }
 
 int qat_crypto_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
diff --git a/drivers/net/i40e/i40e_rxtx.c b/drivers/net/i40e/i40e_rxtx.c
index d42c23c..f571e79 100644
--- a/drivers/net/i40e/i40e_rxtx.c
+++ b/drivers/net/i40e/i40e_rxtx.c
@@ -2221,12 +2221,8 @@ i40e_memzone_reserve(const char *name, uint32_t len, int 
socket_id)
if (mz)
return mz;
 
-   if (rte_xen_dom0_supported())
-   mz = rte_memzone_reserve_bounded(name, len,
-   socket_id, 0, I40E_RING_BASE_ALIGN, 
RTE_PGSIZE_2M);
-   else
-   mz = rte_memzone_reserve_aligned(name, len,
-   socket_id, 0, I40E_RING_BASE_ALIGN);
+   mz = rte_memzone_reserve_aligned(name, len,
+socket_id, 0, I40E_RING_BASE_ALIGN);
return mz;
 }
 
diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index a88916f..e8f7295 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -2835,12 +2835,7 @@ rte_eth_dma_zone_reserve(const struct rte_eth_dev *dev, 
const char *ring_name,
if (mz)
return mz;
 
-   if (rte_xen_dom0_supported())
-   return rte_memzone_reserve_bounded(z_name, size, socket_id,
-  0, align, RTE_PGSIZE_2M);
-   else
-   return rte_memzone_reserve_aligned(z_name, size, socket_id,
-  0, align);
+   return rte_memzone_reserve_aligned(z_name, size, socket_id, 0, align);
 }
 
 int
diff --git a/lib/librte_mempool/rte_mempool.c b/lib/librte_mempool/rte_mempool.c
index 6fc3c9c..6d726ae 100644
--- a/lib/librte_mempool/rte_mempool.c
+++ b/lib/librte_mempool/rte_mempool.c
@@ -527,11 +527,7 @@ rte_mempool_populate_default(struct rte_mempool *mp)
if (mp->nb_mem_chunks != 0)
return -EEXIST;
 
-   if (rte_xen_dom0_supported()) {
-   pg_sz = RTE_PGSIZE_2M;
-   pg_shift = rte_bsf32(pg_sz);
-   align = pg_sz;
-   } else if (rte_eal_has_hugepages()) {
+   if (rte_eal_has_hugepages()) {
pg_shift = 0; /* not needed, zone is physically contiguous */
pg_sz = 0;
align = RTE_CACHE_LINE_SIZE;
@@ -568,7 +564,7 @@ rte_mempool_populate_default(struct rte_mempool *mp)
else
paddr = mz->phys_addr;
 
-   if (rte_eal_has_hugepages() && !rte_xen_dom0_supported())
+   if (rte_eal_has_hugepages())
ret = rte_mempool_populate_phys(mp, mz->addr,
paddr, mz->len,
rte_mempool_memchunk_mz_free,
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 2/6] net/xenvirt: remove

[Jianfeng Tan]

Signed-off-by: Jianfeng Tan 
Acked-by: Bruce Richardson 
---
 MAINTAINERS |   2 -
 app/test-pmd/Makefile   |   4 -
 app/test-pmd/testpmd.c  |  51 +-
 config/common_base  |   5 -
 config/defconfig_arm-armv7a-linuxapp-gcc|   1 -
 doc/guides/nics/features/xenvirt.ini|   6 -
 drivers/net/Makefile|   2 -
 drivers/net/xenvirt/Makefile|  57 --
 drivers/net/xenvirt/rte_eth_xenvirt.c   | 766 
 drivers/net/xenvirt/rte_eth_xenvirt.h   |  61 --
 drivers/net/xenvirt/rte_eth_xenvirt_version.map |   7 -
 drivers/net/xenvirt/rte_mempool_gntalloc.c  | 295 -
 drivers/net/xenvirt/rte_xen_lib.c   | 454 --
 drivers/net/xenvirt/rte_xen_lib.h   | 116 
 drivers/net/xenvirt/virtio_logs.h   |  70 ---
 drivers/net/xenvirt/virtqueue.h | 273 -
 mk/rte.app.mk   |   1 -
 pkg/dpdk.spec   |   3 -
 18 files changed, 18 insertions(+), 2156 deletions(-)
 delete mode 100644 doc/guides/nics/features/xenvirt.ini
 delete mode 100644 drivers/net/xenvirt/Makefile
 delete mode 100644 drivers/net/xenvirt/rte_eth_xenvirt.c
 delete mode 100644 drivers/net/xenvirt/rte_eth_xenvirt.h
 delete mode 100644 drivers/net/xenvirt/rte_eth_xenvirt_version.map
 delete mode 100644 drivers/net/xenvirt/rte_mempool_gntalloc.c
 delete mode 100644 drivers/net/xenvirt/rte_xen_lib.c
 delete mode 100644 drivers/net/xenvirt/rte_xen_lib.h
 delete mode 100644 drivers/net/xenvirt/virtio_logs.h
 delete mode 100644 drivers/net/xenvirt/virtqueue.h

diff --git a/MAINTAINERS b/MAINTAINERS
index fe6c6db..003e72e 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -194,9 +194,7 @@ M: Jianfeng Tan 
 F: lib/librte_eal/linuxapp/xen_dom0/
 F: lib/librte_eal/linuxapp/eal/*xen*
 F: lib/librte_eal/linuxapp/eal/include/exec-env/rte_dom0_common.h
-F: drivers/net/xenvirt/
 F: doc/guides/xen/
-F: doc/guides/nics/features/xenvirt.ini
 
 FreeBSD EAL (with overlaps)
 M: Bruce Richardson 
diff --git a/app/test-pmd/Makefile b/app/test-pmd/Makefile
index c36be19..b6e80dd 100644
--- a/app/test-pmd/Makefile
+++ b/app/test-pmd/Makefile
@@ -77,10 +77,6 @@ ifeq ($(CONFIG_RTE_LIBRTE_BNXT_PMD),y)
 LDLIBS += -lrte_pmd_bnxt
 endif
 
-ifeq ($(CONFIG_RTE_LIBRTE_PMD_XENVIRT),y)
-LDLIBS += -lrte_pmd_xenvirt
-endif
-
 endif
 
 CFLAGS_cmdline.o := -D_GNU_SOURCE
diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index e097ee0..c92fff2 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -76,9 +76,6 @@
 #ifdef RTE_LIBRTE_IXGBE_PMD
 #include 
 #endif
-#ifdef RTE_LIBRTE_PMD_XENVIRT
-#include 
-#endif
 #ifdef RTE_LIBRTE_PDUMP
 #include 
 #endif
@@ -497,37 +494,25 @@ mbuf_pool_create(uint16_t mbuf_seg_size, unsigned nb_mbuf,
"create a new mbuf pool <%s>: n=%u, size=%u, socket=%u\n",
pool_name, nb_mbuf, mbuf_seg_size, socket_id);
 
-#ifdef RTE_LIBRTE_PMD_XENVIRT
-   rte_mp = rte_mempool_gntalloc_create(pool_name, nb_mbuf, mb_size,
-   (unsigned) mb_mempool_cache,
-   sizeof(struct rte_pktmbuf_pool_private),
-   rte_pktmbuf_pool_init, NULL,
-   rte_pktmbuf_init, NULL,
-   socket_id, 0);
-#endif
-
-   /* if the former XEN allocation failed fall back to normal allocation */
-   if (rte_mp == NULL) {
-   if (mp_anon != 0) {
-   rte_mp = rte_mempool_create_empty(pool_name, nb_mbuf,
-   mb_size, (unsigned) mb_mempool_cache,
-   sizeof(struct rte_pktmbuf_pool_private),
-   socket_id, 0);
-   if (rte_mp == NULL)
-   goto err;
-
-   if (rte_mempool_populate_anon(rte_mp) == 0) {
-   rte_mempool_free(rte_mp);
-   rte_mp = NULL;
-   goto err;
-   }
-   rte_pktmbuf_pool_init(rte_mp, NULL);
-   rte_mempool_obj_iter(rte_mp, rte_pktmbuf_init, NULL);
-   } else {
-   /* wrapper to rte_mempool_create() */
-   rte_mp = rte_pktmbuf_pool_create(pool_name, nb_mbuf,
-   mb_mempool_cache, 0, mbuf_seg_size, socket_id);
+   if (mp_anon != 0) {
+   rte_mp = rte_mempool_create_empty(pool_name, nb_mbuf,
+   mb_size, (unsigned) mb_mempool_cache,
+   sizeof(struct rte_pktmbuf_pool_private),
+   socket_id, 0);
+   if (rte_mp == NULL)
+   goto err;
+
+ 

[Xen-devel] [PATCH v2 5/6] eal: remove API rte_mem_phy2mch

[Jianfeng Tan]

Previously, to get MFN address in dom0, this API is a wrapper to
obtain the "physical address".

As we will removed xen dom0 support, this API is not necessary.

Signed-off-by: Jianfeng Tan 
Acked-by: Bruce Richardson 
---
 doc/guides/rel_notes/release_17_11.rst |  2 ++
 drivers/net/e1000/em_rxtx.c|  4 ++--
 drivers/net/e1000/igb_rxtx.c   |  4 ++--
 drivers/net/fm10k/fm10k_ethdev.c   |  4 ++--
 drivers/net/i40e/i40e_ethdev.c |  2 +-
 drivers/net/i40e/i40e_fdir.c   |  2 +-
 drivers/net/i40e/i40e_rxtx.c   |  8 
 drivers/net/ixgbe/ixgbe_rxtx.c |  4 ++--
 drivers/net/sfc/sfc.c  |  2 +-
 lib/librte_eal/common/include/rte_memory.h | 30 --
 lib/librte_mempool/rte_mempool.c   |  3 ---
 11 files changed, 17 insertions(+), 48 deletions(-)

diff --git a/doc/guides/rel_notes/release_17_11.rst 
b/doc/guides/rel_notes/release_17_11.rst
index 170f4f9..8534947 100644
--- a/doc/guides/rel_notes/release_17_11.rst
+++ b/doc/guides/rel_notes/release_17_11.rst
@@ -110,6 +110,8 @@ API Changes
Also, make sure to start the actual text at the margin.
=
 
+   * ``rte_mem_phy2mch`` was used in xen dom0 to obtain the physical address;
+ remove this API as xen dom0 support was removed.
 
 ABI Changes
 ---
diff --git a/drivers/net/e1000/em_rxtx.c b/drivers/net/e1000/em_rxtx.c
index 31819c5..a0f63a7 100644
--- a/drivers/net/e1000/em_rxtx.c
+++ b/drivers/net/e1000/em_rxtx.c
@@ -1289,7 +1289,7 @@ eth_em_tx_queue_setup(struct rte_eth_dev *dev,
txq->port_id = dev->data->port_id;
 
txq->tdt_reg_addr = E1000_PCI_REG_ADDR(hw, E1000_TDT(queue_idx));
-   txq->tx_ring_phys_addr = rte_mem_phy2mch(tz->memseg_id, tz->phys_addr);
+   txq->tx_ring_phys_addr = tz->phys_addr;
txq->tx_ring = (struct e1000_data_desc *) tz->addr;
 
PMD_INIT_LOG(DEBUG, "sw_ring=%p hw_ring=%p dma_addr=0x%"PRIx64,
@@ -1416,7 +1416,7 @@ eth_em_rx_queue_setup(struct rte_eth_dev *dev,
 
rxq->rdt_reg_addr = E1000_PCI_REG_ADDR(hw, E1000_RDT(queue_idx));
rxq->rdh_reg_addr = E1000_PCI_REG_ADDR(hw, E1000_RDH(queue_idx));
-   rxq->rx_ring_phys_addr = rte_mem_phy2mch(rz->memseg_id, rz->phys_addr);
+   rxq->rx_ring_phys_addr = rz->phys_addr;
rxq->rx_ring = (struct e1000_rx_desc *) rz->addr;
 
PMD_INIT_LOG(DEBUG, "sw_ring=%p hw_ring=%p dma_addr=0x%"PRIx64,
diff --git a/drivers/net/e1000/igb_rxtx.c b/drivers/net/e1000/igb_rxtx.c
index 1c80a2a..0fccb5d 100644
--- a/drivers/net/e1000/igb_rxtx.c
+++ b/drivers/net/e1000/igb_rxtx.c
@@ -1530,7 +1530,7 @@ eth_igb_tx_queue_setup(struct rte_eth_dev *dev,
txq->port_id = dev->data->port_id;
 
txq->tdt_reg_addr = E1000_PCI_REG_ADDR(hw, E1000_TDT(txq->reg_idx));
-   txq->tx_ring_phys_addr = rte_mem_phy2mch(tz->memseg_id, tz->phys_addr);
+   txq->tx_ring_phys_addr = tz->phys_addr;
 
txq->tx_ring = (union e1000_adv_tx_desc *) tz->addr;
/* Allocate software ring */
@@ -1667,7 +1667,7 @@ eth_igb_rx_queue_setup(struct rte_eth_dev *dev,
}
rxq->rdt_reg_addr = E1000_PCI_REG_ADDR(hw, E1000_RDT(rxq->reg_idx));
rxq->rdh_reg_addr = E1000_PCI_REG_ADDR(hw, E1000_RDH(rxq->reg_idx));
-   rxq->rx_ring_phys_addr = rte_mem_phy2mch(rz->memseg_id, rz->phys_addr);
+   rxq->rx_ring_phys_addr = rz->phys_addr;
rxq->rx_ring = (union e1000_adv_rx_desc *) rz->addr;
 
/* Allocate software ring. */
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index e60d3a3..15ea2a5 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -1887,7 +1887,7 @@ fm10k_rx_queue_setup(struct rte_eth_dev *dev, uint16_t 
queue_id,
return -ENOMEM;
}
q->hw_ring = mz->addr;
-   q->hw_ring_phys_addr = rte_mem_phy2mch(mz->memseg_id, mz->phys_addr);
+   q->hw_ring_phys_addr = mz->phys_addr;
 
/* Check if number of descs satisfied Vector requirement */
if (!rte_is_power_of_2(nb_desc)) {
@@ -2047,7 +2047,7 @@ fm10k_tx_queue_setup(struct rte_eth_dev *dev, uint16_t 
queue_id,
return -ENOMEM;
}
q->hw_ring = mz->addr;
-   q->hw_ring_phys_addr = rte_mem_phy2mch(mz->memseg_id, mz->phys_addr);
+   q->hw_ring_phys_addr = mz->phys_addr;
 
/*
 * allocate memory for the RS bit tracker. Enough slots to hold the
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index f12aefa..f30b4f5 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -3769,7 +3769,7 @@ i40e_allocate_dma_mem_d(__attribute__((unused)) struct 
i40e_hw *hw,
 
mem->size = size;
mem->va = mz->addr;
-   mem->pa = rte_mem_phy2mch(mz->memseg_id, mz->phys_addr);
+   

[Xen-devel] [libvirt test] 113394: tolerable all pass - PUSHED

[osstest service owner]

flight 113394 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/113394/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 113350
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail  like 113350
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 113350
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qcow2 12 migrate-support-checkfail  never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass

version targeted for testing:
 libvirt  98931187eefdec6f2dea5cb82ab6d23a3ffa6634
baseline version:
 libvirt  cdecfbed027ab242467580a897a636be82d5d411

Last test of basis   113350  2017-09-12 04:26:56 Z1 days
Testing same since   113394  2017-09-13 04:43:36 Z0 days1 attempts


People who touched revisions under test:
  Cole Robinson 
  John Ferlan 
  Michal Privoznik 
  Peter Krempa 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmpass
 test-amd64-amd64-libvirt-xsm pass
 test-armhf-armhf-libvirt-xsm pass
 test-amd64-i386-libvirt-xsm  pass
 test-amd64-amd64-libvirt pass
 test-armhf-armhf-libvirt pass
 test-amd64-i386-libvirt  pass
 test-amd64-amd64-libvirt-pairpass
 test-amd64-i386-libvirt-pair pass
 test-amd64-i386-libvirt-qcow2pass
 test-armhf-armhf-libvirt-raw pass
 test-amd64-amd64-libvirt-vhd pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=libvirt
+ revision=98931187eefdec6f2dea5cb82ab6d23a3ffa6634
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
 export PERLLIB=.:.
 PERLLIB=.:.
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push libvirt 

Re: [Xen-devel] Restore PVHVM DomU with a different MAC

[Kangjie Xi]

2017-09-13 17:24 GMT+08:00 Wei Liu :
> On Wed, Sep 13, 2017 at 04:25:17PM +0800, Kangjie Xi wrote:
>> Hi,
>>
>> I created a Ubuntu-17.04-x64 PVHVM DomU, so I can use "xl
>> network-attach/detach" to add/delete network interfaces, it works
>> well.
>>
>> However I wanted to restore the DomU with a different MAC address, so
>> before saving the checkpoit file, I deleted the network interface with
>> "xl netwrok-detach", and assigned a different MAC in the configuration
>> file and restored the DomU:
>>
>> # xl network-detach ubuntu 0
>> # xl save ubuntu ubuntu.checkpoint
>> # xl restore ubuntu.cfg ubuntu.checkpoint
>>
>
> Doing this causes xl to ignore the saved state and use the provided
> config file to construct the new guest iirc. I haven't checked the code
> in detail though.
>
> The end result seems to be what you wanted.
>
>>  But the DomU didn't detect the new network interface. In Dom0, could
>> see the DomU has new MAC.
>>
>> # xl network-list ubuntu
>> Idx BE Mac Addr. handle state evt-ch   tx-/rx-ring-ref BE-path
>> 0   0  00:16:3e:eb:ca:67 0 1 -1-1/-1
>> /local/domain/0/backend/vif/139/0
>>
>
> A vif is created but DomU can't find it. It is possible DomU kernel
> doesn't go through the whole cycle of device probing and simply ignores
> a vif that it doesn't know about during restore. As far as it is
> concerned, the vif originally provided is already gone. The new vif
> shows up out of nowhere.
>
> You may be able to manually generate a udev event to make the kernel
> probe the new vif. I don't know.
>
>> If I add network interface manually after restoring, the DomU can
>> detect new network interface automatically.
>>
>> # xl network-attach ubuntu type=vif mac=00:16:3e:eb:ca:68 bridge=virbr0
>>
>> # xl network-list ubuntu
>> Idx BE Mac Addr. handle state evt-ch   tx-/rx-ring-ref BE-path
>> 0   0  00:16:3e:eb:ca:67 0 1 -1-1/-1
>> /local/domain/0/backend/vif/139/0
>> 1   0  00:16:3e:eb:ca:68 1 4 -1   529/528
>> /local/domain/0/backend/vif/139/1
>>
>> Is it a bug?
>>
>
> Can't say for sure at this stage.

Windows with PV drivers DomU can restore with a new MAC address.

# xl network-detach windows 0
# xl save windows windows.checkpoint
# xl restore windows.cfg windows.checkpoint

After restoring, the Windows DomU can detect the new network interface
automatically.

>
>> Thanks
>>
>> ___
>> Xen-devel mailing list
>> Xen-devel@lists.xen.org
>> https://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [xen-unstable test] 113387: tolerable FAIL - PUSHED

[osstest service owner]

flight 113387 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/113387/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds 12 guest-start  fail REGR. vs. 113266

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-localmigrate/x10 fail like 113170
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 113170
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 113266
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail  like 113266
 test-amd64-amd64-xl-rtds 10 debian-install   fail  like 113266
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 113266
 test-amd64-amd64-xl-qemut-ws16-amd64 10 windows-installfail never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 10 windows-installfail never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qcow2 12 migrate-support-checkfail  never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 13 guest-saverestore   fail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-amd64-i386-xl-qemut-ws16-amd64 13 guest-saverestore   fail never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass

version targeted for testing:
 xen  16b1414de91b5a82a0996c67f6db3af7d7e32873
baseline version:
 xen  70892c317fd56064b09a4b0fcaa0781735e64efc

Last test of basis   113266  2017-09-11 02:02:27 Z2 days
Failing since113331  2017-09-11 20:50:03 Z2 days3 attempts
Testing same since   113387  2017-09-12 23:20:09 Z1 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  George Dunlap 
  Jan Beulich 
  Juergen Gross 
  Julien Grall 
  Razvan Cojocaru 
  Tim Deegan 
  Wei Liu 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64-xtf  pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 

Re: [Xen-devel] [PATCH v2 1/5] xen:rtds: towards work conserving RTDS

[Dario Faggioli]

On Fri, 2017-09-01 at 11:58 -0400, Meng Xu wrote:
> Make RTDS scheduler work conserving without breaking the real-time
> guarantees.
> 
> VCPU model:
> Each real-time VCPU is extended to have an extratime flag
> and a priority_level field.
> When a VCPU's budget is depleted in the current period,
> if it has extratime flag set,
> its priority_level will increase by 1 and its budget will be
> refilled;
> othewrise, the VCPU will be moved to the depletedq.
> 
> Scheduling policy is modified global EDF:
> A VCPU v1 has higher priority than another VCPU v2 if
> (i) v1 has smaller priority_leve; or
> (ii) v1 has the same priority_level but has a smaller deadline
> 
> Queue management:
> Run queue holds VCPUs with extratime flag set and VCPUs with
> remaining budget. Run queue is sorted in increasing order of VCPUs
> priorities.
> Depleted queue holds VCPUs which have extratime flag cleared and
> depleted budget.
> Replenished queue is not modified.
> 
Mmm.. didn't we agree about putting a word of explanation of how the
spare bandwidth ends up being distributed (i.e., in a way which is
proportional to the utilization)? Or is it there and it's me that am
not finding it?

> --- a/xen/common/sched_rt.c
> +++ b/xen/common/sched_rt.c 
> @@ -245,6 +258,11 @@ static inline struct list_head *rt_replq(const
> struct scheduler *ops)
>  return _priv(ops)->replq;
>  }
>  
> +static inline bool has_extratime(const struct rt_vcpu *svc)
> +{
> +return (svc->flags & RTDS_extratime) ? 1 : 0;
> +}
> +
'true' and 'false'. But I think

 return svc->flags & RTDS_extratime

is just fine already, without any need for the ?: operator.

The rest of the patch looks fine to me.

Dario
-- 
<> (Raistlin Majere)
-
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R Ltd., Cambridge (UK)

signature.asc
Description: This is a digitally signed message part
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 5/5] docs: enable per-VCPU extratime flag for RTDS

[Dario Faggioli]

On Fri, 2017-09-01 at 11:58 -0400, Meng Xu wrote:
> Revise xl tool use case by adding -e option
> Remove work-conserving from TODO list
> 
> Signed-off-by: Meng Xu 
> 
Reviewed-by: Dario Faggioli 

Regards,
Dario
-- 
<> (Raistlin Majere)
-
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R Ltd., Cambridge (UK)

signature.asc
Description: This is a digitally signed message part
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 3/5] xl: enable per-VCPU extratime flag for RTDS

[Dario Faggioli]

On Fri, 2017-09-01 at 11:58 -0400, Meng Xu wrote:
> Change main_sched_rtds and related output functions to support
> per-VCPU extratime flag.
> 
> Signed-off-by: Meng Xu 
> ---
>  tools/xl/xl_cmdtable.c |  3 ++-
>  tools/xl/xl_sched.c| 56 ++
> 
>
Oh, and this patch must update docs/man/xl.pod.1.in as well.

Dario
-- 
<> (Raistlin Majere)
-
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R Ltd., Cambridge (UK)

signature.asc
Description: This is a digitally signed message part
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 4/5] xentrace: enable per-VCPU extratime flag for RTDS

[Dario Faggioli]

On Fri, 2017-09-01 at 11:58 -0400, Meng Xu wrote:
> Change repl_budget event output for xentrace formats and xenalyze
> 
> Signed-off-by: Meng Xu 
> 
Right, thanks for doing this part too. However, can you merge this
inside patch 1?

> diff --git a/tools/xentrace/formats b/tools/xentrace/formats
> index f39182a..470ac5c 100644
> --- a/tools/xentrace/formats
> +++ b/tools/xentrace/formats
> @@ -75,7 +75,7 @@
>  0x00022801  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:tickle[ cpu = 
> %(1)d ]
>  0x00022802  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:runq_pick [ 
> dom:vcpu = 0x%(1)08x, cur_deadline = 0x%(3)08x%(2)08x, cur_budget = 
> 0x%(5)08x%(4)08x ]
>  0x00022803  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:burn_budget   [ 
> dom:vcpu = 0x%(1)08x, cur_budget = 0x%(3)08x%(2)08x, delta = %(4)d ]
> -0x00022804  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:repl_budget   [ 
> dom:vcpu = 0x%(1)08x, cur_deadline = 0x%(3)08x%(2)08x, cur_budget = 
> 0x%(5)08x%(4)08x ]
> +0x00022804  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:repl_budget   [ 
> dom:vcpu = 0x%(1)08x, priority_level = 0x%(2)08d cur_deadline = 
> 0x%(4)08x%(3)08x, cur_budget = 0x%(6)08x%(5)08x ]
>
Why 0x%(2)08d, and not just %(2)d ?

Have you tested it? What does it print?

Dario
-- 
<> (Raistlin Majere)
-
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R Ltd., Cambridge (UK)

signature.asc
Description: This is a digitally signed message part
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 3/5] xl: enable per-VCPU extratime flag for RTDS

[Dario Faggioli]

On Fri, 2017-09-01 at 11:58 -0400, Meng Xu wrote:
> diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
> index ba0159d..1b03d44 100644
> --- a/tools/xl/xl_cmdtable.c
> +++ b/tools/xl/xl_cmdtable.c
> @@ -272,12 +272,13 @@ struct cmd_spec cmd_table[] = {
>  { "sched-rtds",
>    _sched_rtds, 0, 1,
>    "Get/set rtds scheduler parameters",
> -  "[-d  [-v[=VCPUID/all]] [-p[=PERIOD]] [-b[=BUDGET]]]",
> +  "[-d  [-v[=VCPUID/all]] [-p[=PERIOD]] [-b[=BUDGET]] [-
> e[=EXTRATIME]]]",
>    "-d DOMAIN, --domain=DOMAIN Domain to modify\n"
>    "-v VCPUID/all, --vcpuid=VCPUID/allVCPU to modify or
> output;\n"
>    "   Using '-v all' to modify/output all vcpus\n"
>    "-p PERIOD, --period=PERIOD Period (us)\n"
>    "-b BUDGET, --budget=BUDGET Budget (us)\n"
> +  "-e EXTRATIME, --extratime=EXTRATIME EXTRATIME (1=yes, 0=no)\n"
  Extratime
?

>  },
>  { "domid",
>    _domid, 0, 0,
> diff --git a/tools/xl/xl_sched.c b/tools/xl/xl_sched.c
> index 85722fe..5138012 100644
> --- a/tools/xl/xl_sched.c
> +++ b/tools/xl/xl_sched.c
> @@ -251,7 +251,7 @@ static int sched_rtds_domain_output(
>  libxl_domain_sched_params scinfo;
>  
>  if (domid < 0) {
> -printf("%-33s %4s %9s %9s\n", "Name", "ID", "Period",
> "Budget");
> +printf("%-33s %4s %9s %9s %10s\n", "Name", "ID", "Period",
> "Budget", "Extra time");
>  return 0;
>  }
>  
Can you paste the output of:

xl sched-rtds
xl sched-rtds -d 0
xl sched-rtds -d 0 -v 1
xl sched-rtds -d 0 -v all

with the series applied?

> @@ -785,8 +801,9 @@ int main_sched_rtds(int argc, char **argv)
>  goto out;
>  }
>  if (((v_index > b_index) && opt_b) || ((v_index > p_index) &&
> opt_p)
> -|| p_index != b_index) {
> -fprintf(stderr, "Incorrect number of period and budget\n");
> + || ((v_index > e_index) && opt_e) || p_index != b_index
> + || p_index != e_index || b_index != e_index ) {
>
I don't think you need the `b_indes ! e_index` part. If p==b and p==e,
it's automatically true that b==e.

> @@ -820,7 +837,7 @@ int main_sched_rtds(int argc, char **argv)
>  r = EXIT_FAILURE;
>  goto out;
>  }
> -} else if (!opt_p && !opt_b) {
> +} else if (!opt_p && !opt_b && !opt_e) {
>  /* get per-vcpu rtds scheduling parameters */
>  libxl_vcpu_sched_params scinfo;
>  libxl_vcpu_sched_params_init();
> @@ -852,6 +869,7 @@ int main_sched_rtds(int argc, char **argv)
>  scinfo.vcpus[i].vcpuid = vcpus[i];
>  scinfo.vcpus[i].period = periods[i];
>  scinfo.vcpus[i].budget = budgets[i];
> +scinfo.vcpus[i].extratime = extratimes[i] ? 1 :
> 0;
>  }
>  rc = sched_vcpu_set(domid, );
>  } else { /* set params for all vcpus */
> @@ -860,6 +878,7 @@ int main_sched_rtds(int argc, char **argv)
> xmalloc(sizeof(libxl_sched_params));
>  scinfo.vcpus[0].period = periods[0];
>  scinfo.vcpus[0].budget = budgets[0];
> +scinfo.vcpus[0].extratime = extratimes[0] ? 1 : 0;
>
But does these two hunks mean that if I pass `-e 10`, that is
considered a legal way to enable extratime? Shouldn't we enforce
(either here in xl or in libxl) the value to be 0 or 1 ?

Dario
-- 
<> (Raistlin Majere)
-
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R Ltd., Cambridge (UK)

signature.asc
Description: This is a digitally signed message part
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 2/5] libxl: enable per-VCPU extratime flag for RTDS

[Dario Faggioli]

On Fri, 2017-09-01 at 12:03 -0400, Meng Xu wrote:
> On Fri, Sep 1, 2017 at 11:58 AM, Meng Xu 
> wrote:
> > @@ -705,6 +717,12 @@ static int sched_rtds_domain_set(libxl__gc
> > *gc, uint32_t domid,
> >  sdom.period = scinfo->period;
> >  if (scinfo->budget != LIBXL_DOMAIN_SCHED_PARAM_BUDGET_DEFAULT)
> >  sdom.budget = scinfo->budget;
> > +if (scinfo->extratime !=
> > LIBXL_DOMAIN_SCHED_PARAM_EXTRATIME_DEFAULT) {
> > +if (scinfo->extratime)
> > +sdom.flags |= XEN_DOMCTL_SCHEDRT_extra;
> > +else
> > +sdom.flags &= ~XEN_DOMCTL_SCHEDRT_extra;
> > +}
> >  if (sched_rtds_validate_params(gc, sdom.period, sdom.budget))
> >  return ERROR_INVAL;
> 
> 
> As you mentioned in the comment to the xl patch v1, I used
> LIBXL_DOMAIN_SCHED_PARAM_EXTRATIME_DEFAULT for extratime flag as what
> we did for period and budget. But the way we handle flags is exactly
> the same with the way we handle period and budget.
> 
Mmm... and (since you say 'But') is that a problem?

> I'm ok with what it is in this patch, although I feel that we can
> kill the
>  if (scinfo->extratime != LIBXL_DOMAIN_SCHED_PARAM_EXTRATIME_DEFAULT)
> because LIBXL_DOMAIN_SCHED_PARAM_EXTRATIME_DEFAULT is -1.
> 
No, sorry, I don't understand what you mean here...

Dario
-- 
<> (Raistlin Majere)
-
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R Ltd., Cambridge (UK)

signature.asc
Description: This is a digitally signed message part
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 2/5] libxl: enable per-VCPU extratime flag for RTDS

[Dario Faggioli]

On Fri, 2017-09-01 at 11:58 -0400, Meng Xu wrote:
> Modify libxl_vcpu_sched_params_get/set and sched_rtds_vcpu_get/set
> functions to support per-VCPU extratime flag
> 
> Signed-off-by: Meng Xu 
> 
This patch looks ok to me.

Only one thing: in libxl_types.idl is, when its inside
libxl_domain_sched_params, marked as deprecated. I think it should be
moved out of that.

One (very) minor thing too:

> diff --git a/tools/libxl/libxl_sched.c b/tools/libxl/libxl_sched.c
> index faa604e..b76a29a 100644
> --- a/tools/libxl/libxl_sched.c
> +++ b/tools/libxl/libxl_sched.c
> @@ -558,6 +558,10 @@ static int sched_rtds_vcpu_get_all(libxl__gc
> *gc, uint32_t domid,
>  for (i = 0; i < num_vcpus; i++) {
>  scinfo->vcpus[i].period = vcpus[i].u.rtds.period;
>  scinfo->vcpus[i].budget = vcpus[i].u.rtds.budget;
> +if (vcpus[i].u.rtds.flags & XEN_DOMCTL_SCHEDRT_extra)
> +   scinfo->vcpus[i].extratime = 1;
> +else
> +   scinfo->vcpus[i].extratime = 0;
>
This can be:

 scinfo->vcpus[i].extratime = vcpus[i].u.rtds.flags & XEN_DOMCTL_SCHEDRT_extra 
? 1 : 0

or:

 scinfo->vcpus[i].extratime = !!(vcpus[i].u.rtds.flags & 
XEN_DOMCTL_SCHEDRT_extra);

Regards,
Dario 
-- 
<> (Raistlin Majere)
-
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R Ltd., Cambridge (UK)

signature.asc
Description: This is a digitally signed message part
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [xen-unstable-smoke test] 113427: regressions - FAIL

[osstest service owner]

flight 113427 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/113427/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemuu-debianhvm-i386 15 guest-saverestore.2 fail REGR. vs. 
113384

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  ff7eaa55e5055f65f0af8e18398e600f910bcdb1
baseline version:
 xen  082fc63f20e827eb0229d520b4ebf54140d9b21b

Last test of basis   113384  2017-09-12 23:14:17 Z1 days
Failing since113403  2017-09-13 09:03:32 Z0 days6 attempts
Testing same since   113417  2017-09-13 17:17:25 Z0 days3 attempts


People who touched revisions under test:
  Andrew Cooper 
  Haozhong Zhang 
  Jan Beulich 
  Oleksandr Grytsov 
  Petre Pircalabu 
  Wei Liu 

jobs:
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-amd64-amd64-xl-qemuu-debianhvm-i386 fail
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 341 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [xen-4.8-testing baseline-only test] 72102: regressions - trouble: blocked/broken/fail/pass

[Platform Team regression test user]

This run is configured for baseline tests only.

flight 72102 xen-4.8-testing real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/72102/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-midway6 xen-install   fail REGR. vs. 72083
 test-amd64-amd64-qemuu-nested-intel  7 xen-boot   fail REGR. vs. 72083

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 build-arm64   2 hosts-allocate   broken never pass
 build-arm64-pvops 2 hosts-allocate   broken never pass
 build-arm64-xsm   2 hosts-allocate   broken never pass
 build-arm64   3 capture-logs broken never pass
 build-arm64-pvops 3 capture-logs broken never pass
 build-arm64-xsm   3 capture-logs broken never pass
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop  fail like 72083
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop  fail like 72083
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop fail like 72083
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-amd64-xl-pvh-intel 12 guest-start  fail  never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 10 windows-installfail never pass
 test-amd64-amd64-xl-pvh-amd  12 guest-start  fail   never pass
 build-i386-prev   7 xen-build/dist-test  fail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 10 windows-installfail never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-i386-xl-qemut-ws16-amd64 13 guest-saverestore   fail never pass
 test-amd64-i386-xl-qemut-win10-i386 17 guest-stop  fail never pass
 build-amd64-prev  7 xen-build/dist-test  fail   never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail never pass
 test-amd64-i386-xl-qemuu-win10-i386 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemut-win10-i386 17 guest-stop fail never pass

version targeted for testing:
 xen  36898eb12572f0a1f85cb54d4a9e90afcb6f7045
baseline version:
 xen  5e4598106ed02ae4b43abcb29889969eb12867b7

Last test of basis72083  2017-09-09 11:15:32 Z4 days
Testing same since72102  2017-09-13 14:50:31 Z

[Xen-devel] [xen-unstable-smoke test] 113421: regressions - FAIL

[osstest service owner]

flight 113421 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/113421/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemuu-debianhvm-i386 15 guest-saverestore.2 fail REGR. vs. 
113384

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  ff7eaa55e5055f65f0af8e18398e600f910bcdb1
baseline version:
 xen  082fc63f20e827eb0229d520b4ebf54140d9b21b

Last test of basis   113384  2017-09-12 23:14:17 Z0 days
Failing since113403  2017-09-13 09:03:32 Z0 days5 attempts
Testing same since   113417  2017-09-13 17:17:25 Z0 days2 attempts


People who touched revisions under test:
  Andrew Cooper 
  Haozhong Zhang 
  Jan Beulich 
  Oleksandr Grytsov 
  Petre Pircalabu 
  Wei Liu 

jobs:
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-amd64-amd64-xl-qemuu-debianhvm-i386 fail
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 341 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 2/3] credit2: libxl related changes to add support for runqueue per cpupool.

[anshulmakkar]

Introduces scheduler specific parameter at libxl level which are 
passed on to libxc. eg runqueue for credit2

Signed-off-by: Anshul Makkar 
---
 tools/libxl/libxl.h |  2 +-
 tools/libxl/libxl_cpupool.c | 15 +--
 tools/libxl/libxl_types.idl | 46 ++---
 tools/xl/xl_cpupool.c   | 16 ++--
 4 files changed, 63 insertions(+), 16 deletions(-)

diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 91408b4..6617c64 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -2150,7 +2150,7 @@ int libxl_get_freecpus(libxl_ctx *ctx, libxl_bitmap 
*cpumap);
 int libxl_cpupool_create(libxl_ctx *ctx, const char *name,
  libxl_scheduler sched,
  libxl_bitmap cpumap, libxl_uuid *uuid,
- uint32_t *poolid);
+ uint32_t *poolid, const libxl_scheduler_params 
*sched_param);
 int libxl_cpupool_destroy(libxl_ctx *ctx, uint32_t poolid);
 int libxl_cpupool_rename(libxl_ctx *ctx, const char *name, uint32_t poolid);
 int libxl_cpupool_cpuadd(libxl_ctx *ctx, uint32_t poolid, int cpu);
diff --git a/tools/libxl/libxl_cpupool.c b/tools/libxl/libxl_cpupool.c
index 85b0688..e3ce7b3 100644
--- a/tools/libxl/libxl_cpupool.c
+++ b/tools/libxl/libxl_cpupool.c
@@ -130,7 +130,7 @@ int libxl_get_freecpus(libxl_ctx *ctx, libxl_bitmap *cpumap)
 int libxl_cpupool_create(libxl_ctx *ctx, const char *name,
  libxl_scheduler sched,
  libxl_bitmap cpumap, libxl_uuid *uuid,
- uint32_t *poolid)
+ uint32_t *poolid, const libxl_scheduler_params 
*sched_params)
 {
 GC_INIT(ctx);
 int rc;
@@ -138,6 +138,7 @@ int libxl_cpupool_create(libxl_ctx *ctx, const char *name,
 xs_transaction_t t;
 char *uuid_string;
 uint32_t xcpoolid;
+xc_schedparam_t xc_sched_param; 
 
 /* Accept '0' as 'any poolid' for backwards compatibility */
 if ( *poolid == LIBXL_CPUPOOL_POOLID_ANY
@@ -151,8 +152,18 @@ int libxl_cpupool_create(libxl_ctx *ctx, const char *name,
 GC_FREE;
 return ERROR_NOMEM;
 }
+if (sched_params)
+{
+xc_sched_param.u.sched_credit2.ratelimit_us = 
+
sched_params->u.credit2.ratelimit_us;
+xc_sched_param.u.sched_credit2.runq = sched_params->u.credit2.runqueue;
+xc_sched_param.u.sched_credit.tslice_ms = 
sched_params->u.credit.tslice_ms;
+xc_sched_param.u.sched_credit.ratelimit_us = 
sched_params->u.credit.ratelimit_us; 
+}
+else 
+xc_sched_param.u.sched_credit2.runq = LIBXL_CREDIT2_RUNQUEUE_DEFAULT; 
 
-rc = xc_cpupool_create(ctx->xch, , sched);
+rc = xc_cpupool_create(ctx->xch, , sched, _sched_param);
 if (rc) {
 LOGEV(ERROR, rc, "Could not create cpupool");
 GC_FREE;
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index 173d70a..f25429d 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -194,6 +194,16 @@ libxl_scheduler = Enumeration("scheduler", [
 (9, "null"),
 ])
 
+# consistent with sched_credit2.c
+libxl_credit2_runqueue = Enumeration("credit2_runqueue", [
+(0, "CPU"),
+(1, "CORE"),
+(2, "SOCKET"),
+(3, "NODE"),
+(4, "ALL"),
+(5, "DEFAULT"),
+])
+
 # Consistent with SHUTDOWN_* in sched.h (apart from UNKNOWN)
 libxl_shutdown_reason = Enumeration("shutdown_reason", [
 (-1, "unknown"),
@@ -326,15 +336,38 @@ libxl_dominfo = Struct("dominfo",[
 ("domain_type", libxl_domain_type),
 ], dir=DIR_OUT)
 
+libxl_sched_credit_params = Struct("sched_credit_params", [
+("tslice_ms", integer),
+("ratelimit_us", integer),
+], dispose_fn=None)
+
+libxl_sched_credit2_params = Struct("sched_credit2_params", [
+("ratelimit_us", integer),
+("runqueue", libxl_credit2_runqueue),
+], dispose_fn=None)
+ 
+libxl_scheduler_params = Struct("scheduler_params", [
+("u", KeyedUnion(None,libxl_scheduler_tpye "scheduler_type",
+  [("credit2", libxl_sched_credit2_params),
+   ("credit", libxl_sched_credit_params),
+   ("null", None),
+   ("arinc653", None),
+   ("rtds", None),
+   ("unknown", None),
+   ("sedf", None),
+  ])),
+ ])
+
 libxl_cpupoolinfo = Struct("cpupoolinfo", [
 ("poolid",  uint32),
 ("pool_name",   string),
 ("sched",   libxl_scheduler),
 ("n_dom",   uint32),
-("cpumap",  libxl_bitmap)
+("cpumap",  libxl_bitmap),
+("sched_param", libxl_scheduler_params),
 ], dir=DIR_OUT)
 
-libxl_channelinfo = Struct("channelinfo", [
+ibxl_channelinfo = Struct("channelinfo", [
 ("backend", string),
 ("backend_id", uint32),
 ("frontend", string),
@@ -910,15 +943,6 @@ libxl_pcitopology = Struct("pcitopology", [
 ("node", uint32),
 ], dir=DIR_OUT)
 

[Xen-devel] [PATCH 1/3] credit2: libxc related changes to add support for runqueue per cpupool.

[anshulmakkar]

libxc receives scheduler specific configuration parametes from 
libxl.

Signed-off-by: Anshul Makkar 
---
 tools/libxc/include/xenctrl.h | 6 +-
 tools/libxc/xc_cpupool.c  | 4 +++-
 tools/python/xen/lowlevel/xc/xc.c | 3 ++-
 3 files changed, 10 insertions(+), 3 deletions(-)

diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 43151cb..e2157e9 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -1077,17 +1077,21 @@ typedef struct xc_cpupoolinfo {
 
 #define XC_CPUPOOL_POOLID_ANY 0x
 
+typedef xen_sysctl_sched_param_t xc_schedparam_t;
+
 /**
  * Create a new cpupool.
  *
  * @parm xc_handle a handle to an open hypervisor interface
  * @parm ppoolid pointer to the new cpupool id (in/out)
  * @parm sched_id id of scheduler to use for pool
+ * @parm sched_param parameter of the scheduler of the cpupool eg. runq for 
credit2
  * return 0 on success, -1 on failure
  */
 int xc_cpupool_create(xc_interface *xch,
   uint32_t *ppoolid,
-  uint32_t sched_id);
+  uint32_t sched_id,
+  xc_schedparam_t * sched_param);
 
 /**
  * Destroy a cpupool. Pool must be unused and have no cpu assigned.
diff --git a/tools/libxc/xc_cpupool.c b/tools/libxc/xc_cpupool.c
index fbd8cc9..fb2d183 100644
--- a/tools/libxc/xc_cpupool.c
+++ b/tools/libxc/xc_cpupool.c
@@ -36,7 +36,8 @@ static int do_sysctl_save(xc_interface *xch, struct 
xen_sysctl *sysctl)
 
 int xc_cpupool_create(xc_interface *xch,
   uint32_t *ppoolid,
-  uint32_t sched_id)
+  uint32_t sched_id,
+  xc_schedparam_t * sched_params)
 {
 int err;
 DECLARE_SYSCTL;
@@ -46,6 +47,7 @@ int xc_cpupool_create(xc_interface *xch,
 sysctl.u.cpupool_op.cpupool_id = (*ppoolid == XC_CPUPOOL_POOLID_ANY) ?
 XEN_SYSCTL_CPUPOOL_PAR_ANY : *ppoolid;
 sysctl.u.cpupool_op.sched_id = sched_id;
+sysctl.u.cpupool_op.sched_param = *sched_params;
 if ( (err = do_sysctl_save(xch, )) != 0 )
 return err;
 
diff --git a/tools/python/xen/lowlevel/xc/xc.c 
b/tools/python/xen/lowlevel/xc/xc.c
index aa9f8e4..a83a23f 100644
--- a/tools/python/xen/lowlevel/xc/xc.c
+++ b/tools/python/xen/lowlevel/xc/xc.c
@@ -1704,6 +1704,7 @@ static PyObject *pyxc_cpupool_create(XcObject *self,
  PyObject *kwds)
 {
 uint32_t cpupool = XC_CPUPOOL_POOLID_ANY, sched = XEN_SCHEDULER_CREDIT;
+xc_schedparam_t param;
 
 static char *kwd_list[] = { "pool", "sched", NULL };
 
@@ -1711,7 +1712,7 @@ static PyObject *pyxc_cpupool_create(XcObject *self,
   ))
 return NULL;
 
-if ( xc_cpupool_create(self->xc_handle, , sched) < 0 )
+if ( xc_cpupool_create(self->xc_handle, , sched, ) < 0 )
 return pyxc_error_to_exception(self->xc_handle);
 
 return PyLongOrInt_FromLong(cpupool);
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 3/3] credit2: xen related changes to add support for runqueue per cpupool.

[anshulmakkar]

Handles extra scheduler configuration paramers received
at init stage.

Signed-off-by: Anshul Makkar 
---
 xen/common/cpupool.c| 13 -
 xen/common/sched_arinc653.c |  2 +-
 xen/common/sched_credit.c   |  2 +-
 xen/common/sched_credit2.c  |  8 +++-
 xen/common/sched_null.c |  3 ++-
 xen/common/sched_rt.c   |  2 +-
 xen/common/schedule.c   | 11 +++
 xen/include/public/sysctl.h | 45 +
 xen/include/xen/sched-if.h  |  3 ++-
 xen/include/xen/sched.h |  4 +++-
 10 files changed, 61 insertions(+), 32 deletions(-)

diff --git a/xen/common/cpupool.c b/xen/common/cpupool.c
index 9998394..31bace4 100644
--- a/xen/common/cpupool.c
+++ b/xen/common/cpupool.c
@@ -129,12 +129,13 @@ void cpupool_put(struct cpupool *pool)
  * - unknown scheduler
  */
 static struct cpupool *cpupool_create(
-int poolid, unsigned int sched_id, int *perr)
+int poolid, unsigned int sched_id,
+xen_sysctl_sched_param_t param,
+int *perr)
 {
 struct cpupool *c;
 struct cpupool **q;
 int last = 0;
-
 *perr = -ENOMEM;
 if ( (c = alloc_cpupool_struct()) == NULL )
 return NULL;
@@ -171,7 +172,7 @@ static struct cpupool *cpupool_create(
 }
 else
 {
-c->sched = scheduler_alloc(sched_id, perr);
+c->sched = scheduler_alloc(sched_id, param, perr);
 if ( c->sched == NULL )
 {
 spin_unlock(_lock);
@@ -600,10 +601,11 @@ int cpupool_do_sysctl(struct xen_sysctl_cpupool_op *op)
 case XEN_SYSCTL_CPUPOOL_OP_CREATE:
 {
 int poolid;
+xen_sysctl_sched_param_t param = op->sched_param;
 
 poolid = (op->cpupool_id == XEN_SYSCTL_CPUPOOL_PAR_ANY) ?
 CPUPOOLID_NONE: op->cpupool_id;
-c = cpupool_create(poolid, op->sched_id, );
+c = cpupool_create(poolid, op->sched_id, param, );
 if ( c != NULL )
 {
 op->cpupool_id = c->cpupool_id;
@@ -798,7 +800,8 @@ static int __init cpupool_presmp_init(void)
 {
 int err;
 void *cpu = (void *)(long)smp_processor_id();
-cpupool0 = cpupool_create(0, 0, );
+xen_sysctl_sched_param_t param;
+cpupool0 = cpupool_create(0, 0, param, );
 BUG_ON(cpupool0 == NULL);
 cpupool_put(cpupool0);
 cpu_callback(_nfb, CPU_ONLINE, cpu);
diff --git a/xen/common/sched_arinc653.c b/xen/common/sched_arinc653.c
index 0b1b849..1f5d0d0 100644
--- a/xen/common/sched_arinc653.c
+++ b/xen/common/sched_arinc653.c
@@ -343,7 +343,7 @@ arinc653_sched_get(
  *  
  */
 static int
-a653sched_init(struct scheduler *ops)
+a653sched_init(struct scheduler *ops, xen_sysctl_sched_param_t sched_param)
 {
 a653sched_priv_t *prv;
 
diff --git a/xen/common/sched_credit.c b/xen/common/sched_credit.c
index 4fdaa08..44ceaf7 100644
--- a/xen/common/sched_credit.c
+++ b/xen/common/sched_credit.c
@@ -2160,7 +2160,7 @@ csched_dump(const struct scheduler *ops)
 }
 
 static int
-csched_init(struct scheduler *ops)
+csched_init(struct scheduler *ops, xen_sysctl_sched_param_t sched_param)
 {
 struct csched_private *prv;
 
diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c
index 0f93ad5..9b5bbbf 100644
--- a/xen/common/sched_credit2.c
+++ b/xen/common/sched_credit2.c
@@ -391,6 +391,7 @@ struct csched2_private {
 unsigned int load_precision_shift; /* Precision of load calculations */
 unsigned int load_window_shift;/* Lenght of load decaying window */
 unsigned int ratelimit_us; /* Rate limiting for this scheduler   */
+unsigned runqueue; /* cpupool has its own type of runq */
 
 cpumask_t active_queues;   /* Runqueues with (maybe) active cpus */
 struct csched2_runqueue_data *rqd; /* Data of the various runqueues  */
@@ -3349,7 +3350,7 @@ csched2_deinit_pdata(const struct scheduler *ops, void 
*pcpu, int cpu)
 }
 
 static int
-csched2_init(struct scheduler *ops)
+csched2_init(struct scheduler *ops, xen_sysctl_sched_param_t sched_param)
 {
 int i;
 struct csched2_private *prv;
@@ -3410,6 +3411,11 @@ csched2_init(struct scheduler *ops)
 /* initialize ratelimit */
 prv->ratelimit_us = sched_ratelimit_us;
 
+/* not need of type checking here if sched_para.type = credit2. Code
+ * block is here means we have type as credit2.
+ */
+prv->runqueue = sched_param.u.sched_credit2.runq;
+
 prv->load_precision_shift = opt_load_precision_shift;
 prv->load_window_shift = opt_load_window_shift - LOADAVG_GRANULARITY_SHIFT;
 ASSERT(opt_load_window_shift > 0);
diff --git a/xen/common/sched_null.c b/xen/common/sched_null.c
index b4a24ba..cab45c7 100644
--- a/xen/common/sched_null.c
+++ b/xen/common/sched_null.c
@@ -135,7 +135,8 @@ static inline bool vcpu_check_affinity(struct vcpu *v, 
unsigned int cpu,
 return cpumask_test_cpu(cpu, cpumask_scratch_cpu(cpu));
 }
 
-static int null_init(struct scheduler *ops)
+static int 

[Xen-devel] implement runqueue per cpupool

[anshulmakkar]

Attached patch series introduces the concept of  runqueue
per cpupool.

"runqueue" configuration option can be specified with xl command
while configuring cpupool. This will define the basis for 
grouping of cpus (cpu, core, socket, all) in that cpupool.

Series is combined of following patches:
[PATCH 1/3]: libxc related changes to add support for runqueue per cpupool
[PATCH 2/3]: libxl related changes to add support for runqueue per cpupool
[PATCH 3/3]: xen related changes.

On similar lines we can also have runqueue per pool configuration parameter
for Credit scheduler. I plan to send a separate patch series for credit
specific implementation.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [stage1-xen PATCH v1 04/10] build/fedora: Add `run` and `components/*` scripts

[Stefano Stabellini]

On Wed, 13 Sep 2017, Rajiv Ranganath wrote:
> On Tue, Sep 12 2017 at 01:36:04 AM, Stefano Stabellini 
>  wrote:
> 
> [...]
> 
> > Fortunately, from the stage1-xen code point of view, there is very
> > little difference between PVHv2 and PV. Switching from one to the
> > other should be a matter of adding one line to the xl config file.
> 
> There is a related use-case here that I think will be important to
> users.
> 
> In stage1-xen we are packaging a Dom-U kernel. When this kernel crashes
> we would want to capture its crash log. Depending on the nature of the
> issue, users can then work with their own kernel team, vendor (who is
> open to supporting LTS kernels) or upstream.
> 
> We might also want to consider supporting two LTS kernel versions on a
> rolling basis. Users can then use something like labels [1] or
> annotations [2] to toggle the kernel version. That way if their
> containers start crashing under a newer Dom-U kernel, they can roll back
> to a working kernel.
> 
> [...]

Yes, I agree. I think it makes sense to allow users to change the DomU
kernel version, at least at build time. At runtime it would be useful
too. One day we might even support kernels other than Linux.



> > You have a good point. I think we should be clear about the stability
> > of the project and the backward compatibility in the README. We should
> > openly say that it is still a "preview" and there is no "support" or
> > "compatibility" yet.
> 
> Sounds good. I'll update README to reflect this.
>
> > Choosing Xen 4.9 should not be seen as a statement of support. I think
> > we should choose the Xen version based only on the technical merits.
> >
> > In the long term it would be great to support multiple stable versions
> > and a development version of Xen. As of now, I think it makes sense to
> > have an "add-hoc approach": I would use Xen 4.9 just because it is the
> > best choice at the moment. Then, I would update to other versions when
> > it makes sense, manually. I don't think that building against a changing
> > target ("master") is a good idea, because we might end up stumbling
> > across confusing and time-consuming bugs that have nothing to do with
> > stage1-xen. However, we could pick a random commit on the Xen tree if
> > that's convenient for us, because at this stage there is no support
> > really. For example, PVCalls will require some tools changes in Xen.
> > Once they are upstream, we'll want to update the Xen version to the
> > latest with PVCalls support.
> >
> > Does it make sense?
> 
> Yes, it does. I'll switch to xen-4.9, qemu-2.10 and rkt-1.28 in the next
> version of the patchset.

Thanks!


> Best,
> Rajiv
> 
> [1] https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
> [2] 
> https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 7/7] xen/arm: Limit the scope of cpregs.h

[Stefano Stabellini]

On Wed, 13 Sep 2017, Julien Grall wrote:
> Hi Stefano,
> 
> On 09/12/2017 10:53 PM, Stefano Stabellini wrote:
> > On Tue, 12 Sep 2017, Julien Grall wrote:
> > > Currently, cpregs.h is included in pretty much every files even for
> > > arm64. However, the only use for arm64 is when emulating co-processors.
> > > 
> > > For arm32, all users of processor.h expect cpregs.h to be included in
> > > order to access co-processors. So move the inclusion in
> > > asm-arm/arm32/processor.h.
> > > 
> > > cpregs.h will also be directly included in the co-processors emulation
> > > to accommodate arm64.
> > > 
> > > Signed-off-by: Julien Grall 
> > 
> > I can see that the patch works and does what you describe, but what is
> > the benefit? OK, we remove #include  from
> > asm-arm/processor.h, but then we have to add it to vcpreg.c, vgic-v3.c,
> > vtimer.c, and arm32/processor.h. Is there a long term benefit? What
> > prompted you into writing this patch?
> 
> asm/processor.h is included indirectly by every single file of the source
> code. It seems that we use it as a placeholder for anything rather than
> properly splitting in different headers. For instance it contains all the
> definitions of the registers, the traps vector, SMC call, traps prototype...
> For most of those stuff only a limited of files really care about it. So we
> are polluting the name space of each file for no real benefits.
> 
> On AArch64, cpregs.h is only useful when emulating co-processor gregisters. So
> there are no point to include it in a main header that will be pulled by 100s
> files just for the benetifs of 4 files.
> 
> This patch is only a first attempt of clean-up processor.h. Ideally we should
> have more patch to split it.

All right.

Acked-by: Stefano Stabellini 

Maybe add a couple of lines to the commit description saying that by
removing the inclusion of cpregs.h from asm/processor.h we drastically
reduce the exposure of cpregs.h to any source files on ARM64.


> > > ---
> > >  Changes in v2:
> > >  - Update commit message
> > > ---
> > >   xen/arch/arm/smp.c| 1 -
> > >   xen/arch/arm/vcpreg.c | 1 +
> > >   xen/arch/arm/vgic-v3.c| 1 +
> > >   xen/arch/arm/vtimer.c | 2 ++
> > >   xen/include/asm-arm/arm32/processor.h | 2 ++
> > >   xen/include/asm-arm/percpu.h  | 1 -
> > >   xen/include/asm-arm/processor.h   | 1 -
> > >   7 files changed, 6 insertions(+), 3 deletions(-)
> > > 
> > > diff --git a/xen/arch/arm/smp.c b/xen/arch/arm/smp.c
> > > index e7df0874d6..554f4992e6 100644
> > > --- a/xen/arch/arm/smp.c
> > > +++ b/xen/arch/arm/smp.c
> > > @@ -1,6 +1,5 @@
> > >   #include 
> > >   #include 
> > > -#include 
> > >   #include 
> > >   #include 
> > >   #include 
> > > diff --git a/xen/arch/arm/vcpreg.c b/xen/arch/arm/vcpreg.c
> > > index f3b08403fb..e363183ba8 100644
> > > --- a/xen/arch/arm/vcpreg.c
> > > +++ b/xen/arch/arm/vcpreg.c
> > > @@ -18,6 +18,7 @@
> > > #include 
> > >   +#include 
> > >   #include 
> > >   #include 
> > >   #include 
> > > diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
> > > index cbeac28b28..a0cf993d13 100644
> > > --- a/xen/arch/arm/vgic-v3.c
> > > +++ b/xen/arch/arm/vgic-v3.c
> > > @@ -26,6 +26,7 @@
> > >   #include 
> > >   #include 
> > >   +#include 
> > >   #include 
> > >   #include 
> > >   #include 
> > > diff --git a/xen/arch/arm/vtimer.c b/xen/arch/arm/vtimer.c
> > > index 9c7e8f441c..0460962f08 100644
> > > --- a/xen/arch/arm/vtimer.c
> > > +++ b/xen/arch/arm/vtimer.c
> > > @@ -22,6 +22,7 @@
> > >   #include 
> > >   #include 
> > >   +#include 
> > >   #include 
> > >   #include 
> > >   #include 
> > > @@ -29,6 +30,7 @@
> > >   #include 
> > >   #include 
> > >   #include 
> > > +#include 
> > > /*
> > >* Check if regs is allowed access, user_gate is tail end of a
> > > diff --git a/xen/include/asm-arm/arm32/processor.h
> > > b/xen/include/asm-arm/arm32/processor.h
> > > index 68cc82147e..fb330812af 100644
> > > --- a/xen/include/asm-arm/arm32/processor.h
> > > +++ b/xen/include/asm-arm/arm32/processor.h
> > > @@ -1,6 +1,8 @@
> > >   #ifndef __ASM_ARM_ARM32_PROCESSOR_H
> > >   #define __ASM_ARM_ARM32_PROCESSOR_H
> > >   +#include 
> > > +
> > >   #define ACTLR_CAXX_SMP  (1<<6)
> > > #ifndef __ASSEMBLY__
> > > diff --git a/xen/include/asm-arm/percpu.h b/xen/include/asm-arm/percpu.h
> > > index 7968532462..cdf64e0f77 100644
> > > --- a/xen/include/asm-arm/percpu.h
> > > +++ b/xen/include/asm-arm/percpu.h
> > > @@ -4,7 +4,6 @@
> > >   #ifndef __ASSEMBLY__
> > > #include 
> > > -#include 
> > >   #if defined(CONFIG_ARM_32)
> > >   # include 
> > >   #elif defined(CONFIG_ARM_64)
> > > diff --git a/xen/include/asm-arm/processor.h
> > > b/xen/include/asm-arm/processor.h
> > > index d791c12c9c..cd45e5f48f 100644
> > > --- a/xen/include/asm-arm/processor.h
> > > +++ b/xen/include/asm-arm/processor.h
> > > @@ -1,7 +1,6 

[Xen-devel] [linux-linus test] 113381: regressions - FAIL

[osstest service owner]

flight 113381 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/113381/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-localmigrate/x10 fail REGR. vs. 
113031

Regressions which are regarded as allowable (not blocking):
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stopfail REGR. vs. 113031

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-rtds   16 guest-start/debian.repeat fail blocked in 113031
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 113031
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 113031
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 113031
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail  like 113031
 test-amd64-amd64-xl-rtds 10 debian-install   fail  like 113031
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 113031
 test-amd64-amd64-xl-qemut-ws16-amd64 10 windows-installfail never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 10 windows-installfail never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qcow2 12 migrate-support-checkfail  never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 13 guest-saverestore   fail never pass
 test-amd64-i386-xl-qemut-ws16-amd64 13 guest-saverestore   fail never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass

version targeted for testing:
 linux52269718dc2cf2585d7a2828f31d46ef46e68000
baseline version:
 linux569dbb88e80deb68974ef6fdd6a13edb9d686261

Last test of basis   113031  2017-09-04 03:35:52 Z9 days
Failing since113041  2017-09-04 16:49:56 Z9 days   15 attempts
Testing same since   113381  2017-09-12 21:26:03 Z0 days1 attempts


1982 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt 

Re: [Xen-devel] [PATCH 09/15] xen/x86: p2m: Use typesafe GFN in p2m_set_entry

[Tamas K Lengyel]

On Wed, Sep 13, 2017 at 11:59 AM, Julien Grall  wrote:
> Signed-off-by: Julien Grall 
>

I guess the rest of the mem_sharing codebase would benefit from moving
to the use gfn_t as well, clearing up some of the gfn conversion stuff
that's needed right now..

Acked-by: Tamas K Lengyel 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 08/15] xen/x86: p2m: Use typesafe gfn for the P2M callbacks get_entry and set_entry

[Julien Grall]

Hi,

On 13/09/2017 20:08, Razvan Cojocaru wrote:

@@ -142,7 +142,7 @@ bool p2m_mem_access_check(paddr_t gpa, unsigned long gla,
   vm_event_request_t **req_ptr)
 {
 struct vcpu *v = current;
-unsigned long gfn = gpa >> PAGE_SHIFT;
+gfn_t gfn = gaddr_to_gfn(gpa);
 struct domain *d = v->domain;
 struct p2m_domain *p2m = NULL;
 mfn_t mfn;
@@ -215,7 +215,7 @@ bool p2m_mem_access_check(paddr_t gpa, unsigned long gla,
 *req_ptr = req;

 req->reason = VM_EVENT_REASON_MEM_ACCESS;
-req->u.mem_access.gfn = gfn;
+req->u.mem_access.gfn = gfn_x(gfn);
 req->u.mem_access.offset = gpa & ((1 << PAGE_SHIFT) - 1);
 if ( npfec.gla_valid )
 {
@@ -247,7 +247,7 @@ int p2m_set_altp2m_mem_access(struct domain *d, struct 
p2m_domain *hp2m,
 unsigned long gfn_l = gfn_x(gfn);


I'd drop gfn_l completely here and just use gfn_x(gfn) in the two places
below where it's used. It would get rid of a line of code in this
function. But I wouldn't hold the patch over this, so if nobody else has
comments it's fine as is.


Technically more clean-up could be done in every functions. If you 
noticed it, I kept the changes minimal because this is already a boring 
but necessary job and I really don't want to explore all the possible 
clean-up in each single function... Feel free to do more clean-up.



 int rc;

-mfn = ap2m->get_entry(ap2m, gfn_l, , _a, 0, NULL, NULL);
+mfn = ap2m->get_entry(ap2m, gfn, , _a, 0, NULL, NULL);

 /* Check host p2m if no valid entry in alternate */
 if ( !mfn_valid(mfn) )
@@ -264,16 +264,16 @@ int p2m_set_altp2m_mem_access(struct domain *d, struct 
p2m_domain *hp2m,
 if ( page_order != PAGE_ORDER_4K )
 {
 unsigned long mask = ~((1UL << page_order) - 1);
-unsigned long gfn2_l = gfn_l & mask;
+gfn_t gfn2 = _gfn(gfn_l & mask);
 mfn_t mfn2 = _mfn(mfn_x(mfn) & mask);

-rc = ap2m->set_entry(ap2m, gfn2_l, mfn2, page_order, t, old_a, 1);
+rc = ap2m->set_entry(ap2m, gfn2, mfn2, page_order, t, old_a, 1);
 if ( rc )
 return rc;
 }
 }

-return ap2m->set_entry(ap2m, gfn_l, mfn, PAGE_ORDER_4K, t, a,
+return ap2m->set_entry(ap2m, gfn, mfn, PAGE_ORDER_4K, t, a,
  (current->domain != d));


If you need to send V2, could you please also indent the last line so
that '(' is under the 'a' in 'ap2m'? You don't have to, but it'd be a
coding style fix coming in on top of this patch.


Will do.


diff --git a/xen/arch/x86/mm/p2m-ept.c b/xen/arch/x86/mm/p2m-ept.c
index 23c0518733..dff214cf7b 100644
--- a/xen/arch/x86/mm/p2m-ept.c
+++ b/xen/arch/x86/mm/p2m-ept.c
@@ -674,11 +674,12 @@ bool_t ept_handle_misconfig(uint64_t gpa)
  * Returns: 0 for success, -errno for failure
  */
 static int
-ept_set_entry(struct p2m_domain *p2m, unsigned long gfn, mfn_t mfn,
+ept_set_entry(struct p2m_domain *p2m, gfn_t gfn_t, mfn_t mfn,
   unsigned int order, p2m_type_t p2mt, p2m_access_t p2ma,
   int sve)
 {
 ept_entry_t *table, *ept_entry = NULL;
+unsigned long gfn = gfn_x(gfn_t);


Should we call this gfn_l, as done above?


 unsigned long gfn_remainder = gfn;
 unsigned int i, target = order / EPT_TABLE_ORDER;
 unsigned long fn_mask = !mfn_eq(mfn, INVALID_MFN) ? (gfn | mfn_x(mfn)) : 
gfn;
@@ -910,11 +911,12 @@ out:

 /* Read ept p2m entries */
 static mfn_t ept_get_entry(struct p2m_domain *p2m,
-   unsigned long gfn, p2m_type_t *t, p2m_access_t* a,
+   gfn_t gfn_t, p2m_type_t *t, p2m_access_t* a,
p2m_query_t q, unsigned int *page_order,
bool_t *sve)
 {
 ept_entry_t *table = 
map_domain_page(_mfn(pagetable_get_pfn(p2m_get_pagetable(p2m;
+unsigned long gfn = gfn_x(gfn_t);


gfn_l here too?


[...]


diff --git a/xen/arch/x86/mm/p2m-pt.c b/xen/arch/x86/mm/p2m-pt.c
index 0e63d6ed11..57878b1886 100644
--- a/xen/arch/x86/mm/p2m-pt.c
+++ b/xen/arch/x86/mm/p2m-pt.c
@@ -479,12 +479,13 @@ int p2m_pt_handle_deferred_changes(uint64_t gpa)

 /* Returns: 0 for success, -errno for failure */
 static int
-p2m_pt_set_entry(struct p2m_domain *p2m, unsigned long gfn, mfn_t mfn,
+p2m_pt_set_entry(struct p2m_domain *p2m, gfn_t gfn_t, mfn_t mfn,
  unsigned int page_order, p2m_type_t p2mt, p2m_access_t p2ma,
  int sve)
 {
 /* XXX -- this might be able to be faster iff current->domain == d */
 void *table;
+unsigned long gfn = gfn_x(gfn_t);


gfn_l?


 unsigned long i, gfn_remainder = gfn;
 l1_pgentry_t *p2m_entry, entry_content;
 /* Intermediate table to free if we're replacing it with a superpage. */
@@ -731,11 +732,12 @@ p2m_pt_set_entry(struct p2m_domain *p2m, unsigned long 
gfn, mfn_t mfn,
 }

 static mfn_t
-p2m_pt_get_entry(struct p2m_domain *p2m, unsigned long gfn,

Re: [Xen-devel] [Xen-API] [RFC] Unicore Subproject Proposal

[Felipe Huici]

Hi Anil,


>From a MirageOS perspective, we'd be happy to switch to something that
>can give us just enough MiniOS for our ocaml-freestanding [1] code to
>boot on Xen.  One requirement from our side is that we need to strip down
>MiniOS to remove even the C xenstore implementation, since we have pure
>OCaml gnt, xenstore and device driver implementations.

This sounds like a good, reasonable target, and it wouldn’t be the first
time we create guests without xenstore code (as presented by team members
in the last Xen summit and as explained in an upcoming SOSP paper).

>  We'd be happy to try out an alpha Unicore and let you know what is in
>excess to our needs as soon as you have something to publish.

That would be great.

>So full support from MirageOS for this initiative!

Thanks for the support!

— Felipe

>regards,
>Anil
>
>[1] https://github.com/mirage/ocaml-freestanding/
>
>
>___
>Xen-devel mailing list
>Xen-devel@lists.xen.org
>https://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 08/15] xen/x86: p2m: Use typesafe gfn for the P2M callbacks get_entry and set_entry

[Razvan Cojocaru]

On 09/13/2017 09:27 PM, Julien Grall wrote:
> Hi Andrew,
> 
> On 09/13/2017 07:22 PM, Andrew Cooper wrote:
>> On 13/09/2017 18:59, Julien Grall wrote:
>>> diff --git a/xen/arch/x86/mm/p2m-pt.c b/xen/arch/x86/mm/p2m-pt.c
>>> index 0e63d6ed11..57878b1886 100644
>>> --- a/xen/arch/x86/mm/p2m-pt.c
>>> +++ b/xen/arch/x86/mm/p2m-pt.c
>>> @@ -479,12 +479,13 @@ int p2m_pt_handle_deferred_changes(uint64_t gpa)
>>>     /* Returns: 0 for success, -errno for failure */
>>>   static int
>>> -p2m_pt_set_entry(struct p2m_domain *p2m, unsigned long gfn, mfn_t mfn,
>>> +p2m_pt_set_entry(struct p2m_domain *p2m, gfn_t gfn_t, mfn_t mfn,
>>>    unsigned int page_order, p2m_type_t p2mt,
>>> p2m_access_t p2ma,
>>>    int sve)
>>>   {
>>>   /* XXX -- this might be able to be faster iff current->domain
>>> == d */
>>>   void *table;
>>> +    unsigned long gfn = gfn_x(gfn_t);
>>>   unsigned long i, gfn_remainder = gfn;
>>>   l1_pgentry_t *p2m_entry, entry_content;
>>>   /* Intermediate table to free if we're replacing it with a
>>> superpage. */
>>> @@ -731,11 +732,12 @@ p2m_pt_set_entry(struct p2m_domain *p2m,
>>> unsigned long gfn, mfn_t mfn,
>>>   }
>>>     static mfn_t
>>> -p2m_pt_get_entry(struct p2m_domain *p2m, unsigned long gfn,
>>> +p2m_pt_get_entry(struct p2m_domain *p2m, gfn_t gfn_t,
>>>    p2m_type_t *t, p2m_access_t *a, p2m_query_t q,
>>>    unsigned int *page_order, bool_t *sve)
>>>   {
>>>   mfn_t mfn;
>>> +    unsigned long gfn = gfn_x(gfn_t);
>>
>> These two are rather risky, because you shadow the gfn_t type with a
>> variable named gfn_t.  I know its ugly, but how about just gfn_ ?
> 
> I can do that. Hopefully in the future both functions will be fully
> typesafe, so gfn_ will completely disappear.

Ah, I had proposed gfn_l for unsigned long gfns - it seems to be the
unspoken convention in the rest of the code, so I think it fits nicely.


Thanks,
Razvan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [xen-unstable-smoke test] 113417: regressions - FAIL

[osstest service owner]

flight 113417 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/113417/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemuu-debianhvm-i386 15 guest-saverestore.2 fail REGR. vs. 
113384

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  ff7eaa55e5055f65f0af8e18398e600f910bcdb1
baseline version:
 xen  082fc63f20e827eb0229d520b4ebf54140d9b21b

Last test of basis   113384  2017-09-12 23:14:17 Z0 days
Failing since113403  2017-09-13 09:03:32 Z0 days4 attempts
Testing same since   113417  2017-09-13 17:17:25 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Haozhong Zhang 
  Jan Beulich 
  Oleksandr Grytsov 
  Petre Pircalabu 
  Wei Liu 

jobs:
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-amd64-amd64-xl-qemuu-debianhvm-i386 fail
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 341 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 08/15] xen/x86: p2m: Use typesafe gfn for the P2M callbacks get_entry and set_entry

[Razvan Cojocaru]

On 09/13/2017 08:59 PM, Julien Grall wrote:
> Signed-off-by: Julien Grall 
> 
> Cc: Jan Beulich 
> Cc: Andrew Cooper 
> Cc: Razvan Cojocaru 
> Cc: Tamas K Lengyel 
> Cc: George Dunlap 
> Cc: Jun Nakajima 
> Cc: Kevin Tian 
> ---
>  xen/arch/x86/hvm/hvm.c|  2 +-
>  xen/arch/x86/mm/mem_access.c  | 19 +--
>  xen/arch/x86/mm/mem_sharing.c |  4 +--
>  xen/arch/x86/mm/p2m-ept.c |  6 ++--
>  xen/arch/x86/mm/p2m-pod.c | 15 +
>  xen/arch/x86/mm/p2m-pt.c  |  6 ++--
>  xen/arch/x86/mm/p2m.c | 77 
> +++
>  xen/include/asm-x86/p2m.h |  4 +--
>  8 files changed, 73 insertions(+), 60 deletions(-)
> 
> diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
> index 6cb903def5..53be8c093c 100644
> --- a/xen/arch/x86/hvm/hvm.c
> +++ b/xen/arch/x86/hvm/hvm.c
> @@ -1787,7 +1787,7 @@ int hvm_hap_nested_page_fault(paddr_t gpa, unsigned 
> long gla,
>  {
>  bool_t sve;
>  
> -p2m->get_entry(p2m, gfn, , , 0, NULL, );
> +p2m->get_entry(p2m, _gfn(gfn), , , 0, NULL, );
>  
>  if ( !sve && altp2m_vcpu_emulate_ve(curr) )
>  {
> diff --git a/xen/arch/x86/mm/mem_access.c b/xen/arch/x86/mm/mem_access.c
> index 9211fc0abe..948e377e69 100644
> --- a/xen/arch/x86/mm/mem_access.c
> +++ b/xen/arch/x86/mm/mem_access.c
> @@ -66,7 +66,7 @@ static int _p2m_get_mem_access(struct p2m_domain *p2m, 
> gfn_t gfn,
>  }
>  
>  gfn_lock(p2m, gfn, 0);
> -mfn = p2m->get_entry(p2m, gfn_x(gfn), , , 0, NULL, NULL);
> +mfn = p2m->get_entry(p2m, gfn, , , 0, NULL, NULL);
>  gfn_unlock(p2m, gfn, 0);
>  
>  if ( mfn_eq(mfn, INVALID_MFN) )
> @@ -142,7 +142,7 @@ bool p2m_mem_access_check(paddr_t gpa, unsigned long gla,
>vm_event_request_t **req_ptr)
>  {
>  struct vcpu *v = current;
> -unsigned long gfn = gpa >> PAGE_SHIFT;
> +gfn_t gfn = gaddr_to_gfn(gpa);
>  struct domain *d = v->domain;
>  struct p2m_domain *p2m = NULL;
>  mfn_t mfn;
> @@ -215,7 +215,7 @@ bool p2m_mem_access_check(paddr_t gpa, unsigned long gla,
>  *req_ptr = req;
>  
>  req->reason = VM_EVENT_REASON_MEM_ACCESS;
> -req->u.mem_access.gfn = gfn;
> +req->u.mem_access.gfn = gfn_x(gfn);
>  req->u.mem_access.offset = gpa & ((1 << PAGE_SHIFT) - 1);
>  if ( npfec.gla_valid )
>  {
> @@ -247,7 +247,7 @@ int p2m_set_altp2m_mem_access(struct domain *d, struct 
> p2m_domain *hp2m,
>  unsigned long gfn_l = gfn_x(gfn);

I'd drop gfn_l completely here and just use gfn_x(gfn) in the two places
below where it's used. It would get rid of a line of code in this
function. But I wouldn't hold the patch over this, so if nobody else has
comments it's fine as is.

>  int rc;
>  
> -mfn = ap2m->get_entry(ap2m, gfn_l, , _a, 0, NULL, NULL);
> +mfn = ap2m->get_entry(ap2m, gfn, , _a, 0, NULL, NULL);
>  
>  /* Check host p2m if no valid entry in alternate */
>  if ( !mfn_valid(mfn) )
> @@ -264,16 +264,16 @@ int p2m_set_altp2m_mem_access(struct domain *d, struct 
> p2m_domain *hp2m,
>  if ( page_order != PAGE_ORDER_4K )
>  {
>  unsigned long mask = ~((1UL << page_order) - 1);
> -unsigned long gfn2_l = gfn_l & mask;
> +gfn_t gfn2 = _gfn(gfn_l & mask);
>  mfn_t mfn2 = _mfn(mfn_x(mfn) & mask);
>  
> -rc = ap2m->set_entry(ap2m, gfn2_l, mfn2, page_order, t, old_a, 
> 1);
> +rc = ap2m->set_entry(ap2m, gfn2, mfn2, page_order, t, old_a, 1);
>  if ( rc )
>  return rc;
>  }
>  }
>  
> -return ap2m->set_entry(ap2m, gfn_l, mfn, PAGE_ORDER_4K, t, a,
> +return ap2m->set_entry(ap2m, gfn, mfn, PAGE_ORDER_4K, t, a,
>   (current->domain != d));

If you need to send V2, could you please also indent the last line so
that '(' is under the 'a' in 'ap2m'? You don't have to, but it'd be a
coding style fix coming in on top of this patch.

>  }
>  
> @@ -295,10 +295,9 @@ static int set_mem_access(struct domain *d, struct 
> p2m_domain *p2m,
>  mfn_t mfn;
>  p2m_access_t _a;
>  p2m_type_t t;
> -unsigned long gfn_l = gfn_x(gfn);
>  
> -mfn = p2m->get_entry(p2m, gfn_l, , &_a, 0, NULL, NULL);
> -rc = p2m->set_entry(p2m, gfn_l, mfn, PAGE_ORDER_4K, t, a, -1);
> +mfn = p2m->get_entry(p2m, gfn, , &_a, 0, NULL, NULL);
> +rc = p2m->set_entry(p2m, gfn, mfn, PAGE_ORDER_4K, t, a, -1);
>  }
>  
>  return rc;
> diff --git a/xen/arch/x86/mm/mem_sharing.c b/xen/arch/x86/mm/mem_sharing.c
> index 12fb9cc51f..4c1ace9b21 100644
> --- a/xen/arch/x86/mm/mem_sharing.c
> +++ b/xen/arch/x86/mm/mem_sharing.c
> @@ -1234,7 +1234,7 

Re: [Xen-devel] CONFIG_SCRUB_DEBUG=y + arm64 + livepatch = Xen BUG at page_alloc.c:738

[Boris Ostrovsky]

On 09/13/2017 02:25 PM, Julien Grall wrote:
> Hi,
> 
> On 09/13/2017 07:05 PM, Boris Ostrovsky wrote:
>> On 09/13/2017 11:32 AM, Konrad Rzeszutek Wilk wrote:
>> Well, that's not a fix. This eliminates the case that something in
>> ARM-specific code (which I haven't tested) accidentally clears
>> _PGC_need_scrub.
>>
>> OK, I think I know what the problem is. You are using
>> CONFIG_SEPARATE_XENHEAP, are you?
> 
> It seems the bug appear on Arm64, so CONFIG_SEPARATE_XENHEAP is not set.
> 
> Note that Arm32 is using separate heap.


For separate heap we will need


diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index b5243fc..9f62ea2 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -2059,7 +2059,7 @@ void free_xenheap_pages(void *v, unsigned int order)

 memguard_guard_range(v, 1 << (order + PAGE_SHIFT));

-free_heap_pages(virt_to_page(v), order, false);
+free_heap_pages(virt_to_page(v), order, scrub_debug);
 }

 #else


If that doesn't help then there are two cases where free_heap_pages is
called with 'false' --- one in alloc_domheap_pages() and the other in
online_page().

Setting one and then the other would further narrow it down.


-boris

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [ovmf test] 113388: regressions - FAIL

[osstest service owner]

flight 113388 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/113388/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm   6 xen-buildfail REGR. vs. 113143
 build-i386-xsm6 xen-buildfail REGR. vs. 113143
 build-i3866 xen-buildfail REGR. vs. 113143
 build-amd64   6 xen-buildfail REGR. vs. 113143

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a

version targeted for testing:
 ovmf 62ef40c4959e7529053c4f0987e5dafc34880691
baseline version:
 ovmf 3281ebb4ae7de2a858c2e7ec4998b7e55be1a4dc

Last test of basis   113143  2017-09-08 06:17:14 Z5 days
Failing since113156  2017-09-08 19:17:56 Z4 days   26 attempts
Testing same since   113375  2017-09-12 15:46:53 Z1 days2 attempts


People who touched revisions under test:
  Bi, Dandan 
  Brijesh Singh 
  Dandan Bi 
  Hegde Nagaraj P 
  hegdenag 
  Jiaxin Wu 
  Laszlo Ersek 
  Paulo Alcantara 
  Star Zeng 
  Thomas Lamprecht 
  Wu Jiaxin 
  Yonghong Zhu 

jobs:
 build-amd64-xsm  fail
 build-i386-xsm   fail
 build-amd64  fail
 build-i386   fail
 build-amd64-libvirt  blocked 
 build-i386-libvirt   blocked 
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64  blocked 



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 1043 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 00/15] xen/x86: Clean-up the PoD code

[Andrew Cooper]

On 13/09/2017 18:59, Julien Grall wrote:
> Hi all
>
> I have been attempting to use the PoD code on ARM (it will be sent in a
> separate series) and spent sometimes to clean-up and switch to typesafe gfn
> the current code.
>
> The PoD code has been tested on ARM (the version is slightly different, mostly
> renaming) and the x86 part as only been built test it.
>
> Cheers,
>
> Cc: Andrew Cooper 
> Cc: George Dunlap 
> Cc: Jan Beulich 
> Cc: Jun Nakajima 
> Cc: Kevin Tian 
> Cc: Razvan Cojocaru 
> Cc: Tamas K Lengyel 
>
> Julien Grall (15):
>   xen/x86: p2m-pod: Clean-up includes
>   xen/x86: p2m-pod: Remove trailing whitespaces
>   xen/x86: p2m-pod: Fix coding style for comments
>   xen/x86: p2m-pod: Fix coding style
>   xen/x86: p2m-pod: Avoid redundant assignments in
> p2m_pod_demand_populate
>   xen/x86: p2m-pod: Clean-up use of typesafe MFN
>   xen/x86: p2m-pod: Use typesafe gfn in p2m_pod_decrease_reservation
>   xen/x86: p2m: Use typesafe gfn for the P2M callbacks get_entry and
> set_entry
>   xen/x86: p2m: Use typesafe GFN in p2m_set_entry
>   xen/x86: p2m-pod: Use typesafe GFN in pod_eager_record
>   xen/x86: p2m-pod: Clean-up p2m_pod_zero_check
>   xen/x86: p2m-pod: Use typesafe gfn in p2m_pod_zero_check
>   xen/x86: p2m-pod: Use typesafe gfn in p2m_pod_demand_populate
>   xen/x86: p2m-pod: Use typesafe gfn for the fields reclaim_single and
> max_guest
>   xen/x86: p2m-pod: Rework prototype of p2m_pod_demand_populate

Only the one minor comment on patch 8, and its knock-on effect in later
patches.

Acked-by: Andrew Cooper  (with either my
suggestion, or a suitable alternative)

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 08/15] xen/x86: p2m: Use typesafe gfn for the P2M callbacks get_entry and set_entry

[Julien Grall]

Hi Andrew,

On 09/13/2017 07:22 PM, Andrew Cooper wrote:

On 13/09/2017 18:59, Julien Grall wrote:

diff --git a/xen/arch/x86/mm/p2m-pt.c b/xen/arch/x86/mm/p2m-pt.c
index 0e63d6ed11..57878b1886 100644
--- a/xen/arch/x86/mm/p2m-pt.c
+++ b/xen/arch/x86/mm/p2m-pt.c
@@ -479,12 +479,13 @@ int p2m_pt_handle_deferred_changes(uint64_t gpa)
  
  /* Returns: 0 for success, -errno for failure */

  static int
-p2m_pt_set_entry(struct p2m_domain *p2m, unsigned long gfn, mfn_t mfn,
+p2m_pt_set_entry(struct p2m_domain *p2m, gfn_t gfn_t, mfn_t mfn,
   unsigned int page_order, p2m_type_t p2mt, p2m_access_t p2ma,
   int sve)
  {
  /* XXX -- this might be able to be faster iff current->domain == d */
  void *table;
+unsigned long gfn = gfn_x(gfn_t);
  unsigned long i, gfn_remainder = gfn;
  l1_pgentry_t *p2m_entry, entry_content;
  /* Intermediate table to free if we're replacing it with a superpage. */
@@ -731,11 +732,12 @@ p2m_pt_set_entry(struct p2m_domain *p2m, unsigned long 
gfn, mfn_t mfn,
  }
  
  static mfn_t

-p2m_pt_get_entry(struct p2m_domain *p2m, unsigned long gfn,
+p2m_pt_get_entry(struct p2m_domain *p2m, gfn_t gfn_t,
   p2m_type_t *t, p2m_access_t *a, p2m_query_t q,
   unsigned int *page_order, bool_t *sve)
  {
  mfn_t mfn;
+unsigned long gfn = gfn_x(gfn_t);


These two are rather risky, because you shadow the gfn_t type with a
variable named gfn_t.  I know its ugly, but how about just gfn_ ?


I can do that. Hopefully in the future both functions will be fully 
typesafe, so gfn_ will completely disappear.


Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Windows "heinsenbug" (WAS: Re: Notes Design Session: Making Releases Lessons Learned: Improving Our Release Process and Tooling)

[Paul Durrant]

> -Original Message-
> From: Xen-devel [mailto:xen-devel-boun...@lists.xen.org] On Behalf Of
> Julien Grall
> Sent: 13 September 2017 10:52
> To: Ian Jackson ; Jan Beuli ch 
> Cc: Juergen Gross ; Wei Liu ;
> ross.philip...@gmail.com; lars.kurth@gmail.com; xen-
> de...@lists.xen.org; committ...@xenproject.org
> Subject: [Xen-devel] Windows "heinsenbug" (WAS: Re: Notes Design
> Session: Making Releases Lessons Learned: Improving Our Release Process
> and Tooling)
> 
> Hi all,
> 
> On 07/27/2017 05:56 PM, Ian Jackson wrote:
> > Jan Beulich writes ("Re: Notes Design Session: Making Releases Lessons
> Learned: Improving Our Release Process and Tooling"):
> >> [Lars:]
> >>> Ian: These are most likely software problems, most likely in Xen.
> >>> ISSUE: nobody wants to debug Windows Heisenbugs
> >>
> >> Hmm, I think we believe to have understood what the issue is, it's just
> that
> >> the change needed is on the tool stack side (gracefully deal with a domain
> >> rebooting while being migrated).
> >
> > No, the problem is that we don't understand why the domain is
> > rebooting.  It should not be.
> 
> This bug still seem to be around and likely going to impact once again
> the release testing.
> 
> As discussed during the summit, I am really thinking to make this
> "heinsenbug" a blocker until somehow finally spend sometimes
> investigating the problem and suggest actions.

If testing was done with PV drivers installed then we could get a lot more 
information into the logs. BSODs vs. clean shutdowns (perhaps due to Windows 
Update) would then become obvious.

  Paul

> 
> Cheers,
> 
> --
> Julien Grall
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] CONFIG_SCRUB_DEBUG=y + arm64 + livepatch = Xen BUG at page_alloc.c:738

[Julien Grall]

Hi,

On 09/13/2017 07:05 PM, Boris Ostrovsky wrote:

On 09/13/2017 11:32 AM, Konrad Rzeszutek Wilk wrote:
Well, that's not a fix. This eliminates the case that something in
ARM-specific code (which I haven't tested) accidentally clears
_PGC_need_scrub.

OK, I think I know what the problem is. You are using
CONFIG_SEPARATE_XENHEAP, are you?


It seems the bug appear on Arm64, so CONFIG_SEPARATE_XENHEAP is not set.

Note that Arm32 is using separate heap.

Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] vTPM Manager VM launch failure: operation not permitted

[Ronny Ko]

Hi Quan,

As you suggested, I recompiled Linux without TPM module (so I don't have
/dev/tpm0 anymore), and added "extra=tpm2" and "iomem=["fed50,5"]. But I
still get the same error:

Parsing config from vtpmmgr.cfg
> libxl: error: libxl_create.c:1297:domcreate_launch_dm: Domain 1:failed
> give domain access to iomem range fed44-fed44: Operation not permitted
> libxl: error: libxl_domain.c:1003:libxl__destroy_domid: Domain
> 1:Non-existant domain
> libxl: error: libxl_domain.c:962:domain_destroy_callback: Domain 1:Unable
> to destroy guest
> libxl: error: libxl_domain.c:889:domain_destroy_cb: Domain 1:Destruction
> of domain failed
>

I get the same error if I try from another computer. Is it that you don't
get this error if you do the same?

Ronny
Wed, Sep 13, 2017 at 11:03 AM

On Wed, Sep 13, 2017 at 11:03 AM, Quan Xu  wrote:

>
> Ronny Ko On 2017/9/13 Wed 22:26 wrote：
>
>> Hi Quan,
>>
>> My phsyical TPM is v2.0. I alrady tried 'iomem=["fed40,1"]' but didn't
>> work..
>>
>> Actually, my DOM's TPM driver has been loaded.
>> Meanwhile, I thought xen-devel was too busy with other real issues, so I
>> asked this question to Daniel after that and he sent me a patch. I am
>> trying out his patch on xen-unstable, and if the patch works Daniel will
>> submit it upstream.
>>
>> I will let you guys know about the result soon.
>>
>
> As the TPM manager uses direct access to the physical TPM, it may
> interfere
>
> to access to the TPM by dom0.
>
>
> for tpm2.0
>
>
>
> Add:
>
> ..
>
>  extra="tpm2"
>
> ..
>
> extra option to launch vtpmmgr domain on TPM 2.0, and ignore it on TPM
>
> 1.x. for example:
>
>
> kernel="/usr/lib/xen/boot/vtpmmgr.gz"
>
> memory=128
>
> disk=["file:/home/vtpm2/vmgr,hda,w"]
>
> name="vtpmmgr"
>
> iomem=["fed40,5"]
>
> extra="tpm2"
>
> try again, good luck
>
> Quan
>
>
>>
>> On Wed, Sep 13, 2017 at 8:27 AM, Quan Xu  wrote:
>>
>>>
>>>
>>> on 2017/9/13 18:42, Wei Liu wrote:
>>>
 Cc VTPM maintainers

 On Sun, Sep 10, 2017 at 03:07:04PM -0400, Ronny Ko wrote:

> Hi,
>
> I'm a PhD student from Harvard University having a trouble in running
> vTPM manager.
>
> I cannot successfully launch vTPM manager in Xen, because when I
> command "sudo xl create vtpm-manager.cfg" to launch a virtual TPM VM,
> I get the following error:
>
> libxl: error: libxl_create.c:1295:domcreate_launch_dm: Domain
> 10:failed give domain access to iomeim range fed44-fed44: Operation
> not permitted
>
> In Xen, virtual TPM is a standalone VM that communicates with DOMu.
> "vtpm-manager.cfg" is Xen's configuration file for virtual TPM manager
> VM, whose contents are as follows:
>
>  vtpm-manager.cfg 
> kernel="/usr/local/lib/xen/boot/vtpmmgr-stubdom.gz"   # vTPM manager
> code image
> memory=16   # 16M RAM size
> disk=["file:/home/skyer/Desktop/xen/vtpmmgr-stubdom.img,hda,w"]   #
> disk storage
> name="vtpmmgr"   # Just a nick name
> iomem=["fed44,1"]   # This means, map physical memory from
> 0xfed44000-0xfed44fff for I/O, which is to be used by virtual TPM
> manager to communicate with the physical TPM device.
> ===
>

>>> Ronny,
>>> is your physical TPM device v1.2 or v2.0?
>>>
>>> for tpm1.2.. , commands that are sent to the TPM through the register
>>> set at address FED4. are implicitly associated with locality 0.
>>> try 'iomem=["fed40,1"]'
>>>
>>>
>>> and make sure Dom0 's TPM driver is _not_ loaded...
>>>
>>> Quan
>>>
>>> My kernel is compiled with CONFIG_IO_STRICT_DEVMEM flag disabled, so
> iomem shouldn't be blocked by the kernel. I tried to map not only
> 0xfed44000, but also any other random addresses for testing, but all
> of them give the same error message as above.
>
> I'm launching the vTPM manager VM not from inside a DOMu Linux VM, but
> from inside the Linux kernel directly loaded by Xen-4.9.0 (which I
> suppose to be DOM0 Linux VM), and I believe this is the correct way to
> launch vTPM manager.
>
> In particular, I get the iomem() "operation not allowed" error at the
> source code line;
> ioctl(fd, IOCTL_PRIVCMD_HYPERCALL, hypercall);
>
> In ioctl(fd, IOCTL_PRIVCMD_HYPERCALL, hypercall),
> - "fd" is the special privileged Command device
> - "IOCTL_PRIVCMD_HYPERCALL" denotes that this is a privileged
> hypercall command
> - "hypercall" is an object containing the information of: {
> hypercall_command_index, target_DOM_id, iomem_start_page,
> iomem_page_count, allow_or_deny_access}.
>
> When I launch the vTPM manager, target_DOM_id = the ID of vTPM
> manager, iomem_start_page = 0xfed40, iomem_page_count = 5, and
> allow_or_deny_access = 1, and this ioctl() gives an
> "operation-not-allowed" error. But if I hard-code 

Re: [Xen-devel] [PATCH 08/15] xen/x86: p2m: Use typesafe gfn for the P2M callbacks get_entry and set_entry

[Andrew Cooper]

On 13/09/2017 18:59, Julien Grall wrote:
> diff --git a/xen/arch/x86/mm/p2m-pt.c b/xen/arch/x86/mm/p2m-pt.c
> index 0e63d6ed11..57878b1886 100644
> --- a/xen/arch/x86/mm/p2m-pt.c
> +++ b/xen/arch/x86/mm/p2m-pt.c
> @@ -479,12 +479,13 @@ int p2m_pt_handle_deferred_changes(uint64_t gpa)
>  
>  /* Returns: 0 for success, -errno for failure */
>  static int
> -p2m_pt_set_entry(struct p2m_domain *p2m, unsigned long gfn, mfn_t mfn,
> +p2m_pt_set_entry(struct p2m_domain *p2m, gfn_t gfn_t, mfn_t mfn,
>   unsigned int page_order, p2m_type_t p2mt, p2m_access_t p2ma,
>   int sve)
>  {
>  /* XXX -- this might be able to be faster iff current->domain == d */
>  void *table;
> +unsigned long gfn = gfn_x(gfn_t);
>  unsigned long i, gfn_remainder = gfn;
>  l1_pgentry_t *p2m_entry, entry_content;
>  /* Intermediate table to free if we're replacing it with a superpage. */
> @@ -731,11 +732,12 @@ p2m_pt_set_entry(struct p2m_domain *p2m, unsigned long 
> gfn, mfn_t mfn,
>  }
>  
>  static mfn_t
> -p2m_pt_get_entry(struct p2m_domain *p2m, unsigned long gfn,
> +p2m_pt_get_entry(struct p2m_domain *p2m, gfn_t gfn_t,
>   p2m_type_t *t, p2m_access_t *a, p2m_query_t q,
>   unsigned int *page_order, bool_t *sve)
>  {
>  mfn_t mfn;
> +unsigned long gfn = gfn_x(gfn_t);

These two are rather risky, because you shadow the gfn_t type with a
variable named gfn_t.  I know its ugly, but how about just gfn_ ?

~Andrew

>  paddr_t addr = ((paddr_t)gfn) << PAGE_SHIFT;
>  l2_pgentry_t *l2e;
>  l1_pgentry_t *l1e;
>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v3 0/1] netif: staging grants for I/O requests

[Joao Martins]

Hey,

This is v3 taking into consideration all comments received from v2 (changelog
in the first patch). The specification is right after the diffstat.

Reference implementation also here (on top of net-next):

https://github.com/jpemartins/linux.git xen-net-stg-gnts-v3

Although I am satisfied with how things are being done above, I wanted
to request some advise/input on whether there could be a simpler way of
achieving the same. Specifically because these control messages
adds up significant code on the frontend to pregrant, and in other cases the
control message might be limitative if frontend tries to keep a dinamically
changed buffer pool in different queues. *Maybe* it could be simpler to adjust
the TX/RX ring ABI in a compatible matter (Disclaimer: I haven't implemented
this just yet):

 1) Add a flag `NETTXF_persist` to `netif_tx_request`

 2) Replace RX `netif_rx_request` padding with `flags` and adda
 `NETRXF_persist` with the same purpose as 1).

 3) This remains backwards compatible as backends not supporting this wouldn't
 act on this new flag, and given we replace padding with flags means unsupported
 backends won't simply be aware of RX *request* `flags`. This is under the
 assumption that there's no requirement that padding must be zero throughout
 the netif.h specification.

 4) Keeping `GET_GREF_MAPPING_SIZE` ctrl msg for frontend to do better
 decisions?

 5) Semantics are simple: slots with flags marked as NET{RX,TX}F_persist
 represent a permanent mapped ref and therefore mapped if non-existent.
 *future* omissions of the flag signals the mapping should be removed.

This would allow guests which reuse buffers (apparently Windows :)) to scale
better as unmaps would be done on the individual queue context  plus allowing
frontend to remain a more simple in the management of "permanent" buffers. The
drawback seems to be the added complexity (and somewhat racy behaviour) on the
datapath, to map or unmap accordingly. Because now we would have to
differentiate between long vs short lived map/unmap ops in addition to looking
up on our mappings table. Thoughts, or perhaps people may prefer the one
already described in the series?

Cheers,

Joao Martins (1):
  public/io/netif.h: add gref mapping control messages

 xen/include/public/io/netif.h | 115 ++
 1 file changed, 115 insertions(+)
---
% Staging grants for network I/O requests
% Joao Martins <>
% Revision 3

\clearpage


Architecture(s): Any


# Background and Motivation

At the Xen hackaton '16 networking session, we spoke about having a permanently
mapped region to describe header/linear region of packet buffers. This document
outlines the proposal covering motivation of this and applicability for other
use-cases alongside the necessary changes. This proposal is an RFC and also
includes alternative solutions.

The motivation of this work is to eliminate grant ops for packet I/O intensive
workloads such as those observed with smaller requests size (i.e. <= 256 bytes
or <= MTU). Currently on Xen, only bulk transfer (e.g. 32K..64K packets) are the
only ones performing really good (up to 80 Gbit/s in few CPUs), usually
backing end-hosts and server appliances. Anything that involves higher packet
rates (<= 1500 MTU) or without sg, performs badly almost like a 1 Gbit/s
throughput.

# Proposal

The proposal is to leverage the already implicit copy from and to packet linear
data on netfront and netback, to be done instead from a permanently mapped
region. In some (physical) NICs this is known as header/data split.

Specifically some workloads (e.g. NFV) it would provide a big increase in
throughput when we switch to (zero)copying in the backend/frontend, instead of
the grant hypercalls. Thus this extension aims at futureproofing the netif
protocol by adding the possibility of guests setting up a list of grants that
are set up at device creation and revoked at device freeing - without taking
too much grant entries in account for the general case (i.e. to cover only the
header region <= 256 bytes, 16 grants per ring) while configurable by kernel
when one wants to resort to a copy-based as opposed to grant copy/map.

\clearpage

# General Operation

Here we describe how netback and netfront general operate, and where the 
proposed
solution will fit. The security mechanism currently involves grants references
which in essence are round-robin recycled 'tickets' stamped with the GPFNs,
permission attributes, and the authorized domain:

(This is an in-memory view of struct grant_entry_v1):

 0 1 2 3 4 5 6 7 octet
++---++
| flags  | domain id | frame  |
++---++

Where there are N grant entries in a grant 

[Xen-devel] [PATCH v3 1/1] public/io/netif.h: add gref mapping control messages

[Joao Martins]

Adds 3 messages to allow guest to let backend keep grants mapped,
such that 1) guests allowing fast recycling of pages can avoid doing
grant ops for those cases, or otherwise 2) preferring copies over
grants and 3) always using a fixed set of pages for network I/O.

The three control ring messages added are:
 - Add grefs to be mapped by backend
 - Remove grefs mappings (If they are not in use)
 - Get maximum amount of grefs kept mapped.

Signed-off-by: Joao Martins 
---
v3:
* Use DEL for unmapping grefs instead of PUT
* Rname from xen_netif_gref_alloc to xen_netif_gref
* Add 'status' field on xen_netif_gref
* Clarify what 'inflight' means
* Use "beginning of the page" instead of "beginning of the grant"
* Mention that page needs to be r/w (as it will have to modify \.status)
* `data` on ADD|PUT returns number of entries mapped/unmapped.
---
 xen/include/public/io/netif.h | 115 ++
 1 file changed, 115 insertions(+)

diff --git a/xen/include/public/io/netif.h b/xen/include/public/io/netif.h
index ca0061410d..0080a260fd 100644
--- a/xen/include/public/io/netif.h
+++ b/xen/include/public/io/netif.h
@@ -353,6 +353,9 @@ struct xen_netif_ctrl_request {
 #define XEN_NETIF_CTRL_TYPE_SET_HASH_MAPPING_SIZE 5
 #define XEN_NETIF_CTRL_TYPE_SET_HASH_MAPPING  6
 #define XEN_NETIF_CTRL_TYPE_SET_HASH_ALGORITHM7
+#define XEN_NETIF_CTRL_TYPE_GET_GREF_MAPPING_SIZE 8
+#define XEN_NETIF_CTRL_TYPE_ADD_GREF_MAPPING  9
+#define XEN_NETIF_CTRL_TYPE_DEL_GREF_MAPPING 10
 
 uint32_t data[3];
 };
@@ -391,6 +394,41 @@ struct xen_netif_ctrl_response {
 };
 
 /*
+ * Static Grants (struct xen_netif_gref)
+ * =
+ *
+ * A frontend may provide a fixed set of grant references to be mapped on
+ * the backend. The message of type XEN_NETIF_CTRL_TYPE_ADD_GREF_MAPPING
+ * prior its usage in the command ring allows for creation of these mappings.
+ * The backend will maintain a fixed amount of these mappings.
+ *
+ * XEN_NETIF_CTRL_TYPE_GET_GREF_MAPPING_SIZE lets a frontend query how many
+ * of these mappings can be kept.
+ *
+ * Each entry in the XEN_NETIF_CTRL_TYPE_{ADD,DEL}_GREF_MAPPING input table has
+ * the following format:
+ *
+ *0 1 2 3 4 5 6 7  octet
+ * +-+-+-+-+-+-+-+-+
+ * | grant ref |  flags|  status   |
+ * +-+-+-+-+-+-+-+-+
+ *
+ * grant ref: grant reference
+ * flags: flags describing the control operation
+ * status: XEN_NETIF_CTRL_STATUS_*
+ */
+
+struct xen_netif_gref {
+   grant_ref_t ref;
+   uint16_t flags;
+
+#define _XEN_NETIF_CTRLF_GREF_readonly0
+#define XEN_NETIF_CTRLF_GREF_readonly(1U<<_XEN_NETIF_CTRLF_GREF_readonly)
+
+   uint16_t status;
+};
+
+/*
  * Control messages
  * 
  *
@@ -609,6 +647,83 @@ struct xen_netif_ctrl_response {
  *   invalidate any table data outside that range.
  *   The grant reference may be read-only and must remain valid until
  *   the response has been processed.
+ *
+ * XEN_NETIF_CTRL_TYPE_GET_GREF_MAPPING_SIZE
+ * -
+ *
+ * This is sent by the frontend to fetch the number of grefs that can be kept
+ * mapped in the backend.
+ *
+ * Request:
+ *
+ *  type= XEN_NETIF_CTRL_TYPE_GET_GREF_MAPPING_SIZE
+ *  data[0] = queue index (assumed 0 for single queue)
+ *  data[1] = 0
+ *  data[2] = 0
+ *
+ * Response:
+ *
+ *  status = XEN_NETIF_CTRL_STATUS_NOT_SUPPORTED - Operation not
+ * supported
+ *   XEN_NETIF_CTRL_STATUS_INVALID_PARAMETER - The queue index is
+ * out of range
+ *   XEN_NETIF_CTRL_STATUS_SUCCESS   - Operation successful
+ *  data   = maximum number of entries allowed in the gref mapping table
+ *   (if operation was successful) or zero if it is not supported.
+ *
+ * XEN_NETIF_CTRL_TYPE_ADD_GREF_MAPPING
+ * 
+ *
+ * This is sent by the frontend for backend to map a list of grant
+ * references.
+ *
+ * Request:
+ *
+ *  type= XEN_NETIF_CTRL_TYPE_ADD_GREF_MAPPING
+ *  data[0] = queue index
+ *  data[1] = grant reference of page containing the mapping list
+ *(r/w and assumed to start at beginning of page)
+ *  data[2] = size of list in entries
+ *
+ * Response:
+ *
+ *  status = XEN_NETIF_CTRL_STATUS_NOT_SUPPORTED - Operation not
+ * supported
+ *   XEN_NETIF_CTRL_STATUS_INVALID_PARAMETER - Operation failed
+ *   XEN_NETIF_CTRL_STATUS_SUCCESS   - Operation successful
+ *  data   = number of entries that were mapped
+ *
+ * NOTE: Each entry in the input table has the format outlined
+ *   in struct xen_netif_gref.
+ *
+ * XEN_NETIF_CTRL_TYPE_DEL_GREF_MAPPING
+ * 

Re: [Xen-devel] CONFIG_SCRUB_DEBUG=y + arm64 + livepatch = Xen BUG at page_alloc.c:738

[Boris Ostrovsky]

On 09/13/2017 11:32 AM, Konrad Rzeszutek Wilk wrote:
> On Tue, Sep 12, 2017 at 09:19:23PM -0400, Boris Ostrovsky wrote:
>>
>> On 09/12/2017 08:01 PM, Konrad Rzeszutek Wilk wrote:
>>> On Mon, Sep 11, 2017 at 08:45:02PM -0400, Boris Ostrovsky wrote:

 On 09/11/2017 07:55 PM, Konrad Rzeszutek Wilk wrote:
> Hey,
>
> I've only been able to reproduce this on ARM64 (trying right now ARM32
> as well), and not on x86.
>
> If I compile Xen without CONFIG_SCRUB_DEBUG it works great. But if
> enable it and try to load a livepatch it blows up in page_alloc.c:738
>
> This is with origin/staging (d0291f3391)
 Can you still reproduce this if you revert 307c3be?
>>> Sadly yes - it still crashes. I didn't capture the serial output.
>>>
>>> I honestly think the issue is that on ARM64 the "sleep" loop does not
>>> wake up as often as on x86 (CC-ing Dariof who I believe observed this
>>> with Credit2 and the wakeup.. something) - maybe he remembers the
>>> details. Anyhow my theory is that the pages are not scrubbed at all
>>> when they go in the idle loop as once it goes to sleep - it stays there.
>>
>> There is no (well, should not be) any timing dependencies in how/whether
>> pages are scrubbed. If a page doesn't get scrubbed because someone didn't
>> wake up then it should be scrubbed in alloc_heap_pages(). So in this case
>> the page is thought to be clean (_PGC_need_scrub is not set), but it is not.
>>
>> Have you tried running a guest (or two), rebooting in a loop?
> No. I just cold-booted it and tried to livepatch.
>> Another thing to try is to set need_scrub to true in free_heap_pages().
> Magic!
>
> diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
> index dbad1e1ca0..9303eb4517 100644
> --- a/xen/common/page_alloc.c
> +++ b/xen/common/page_alloc.c
> @@ -1308,6 +1308,7 @@ static void free_heap_pages(
>  ASSERT(node >= 0);
>  
>  spin_lock(_lock);
> +need_scrub = true;
>  
>  for ( i = 0; i < (1 << order); i++ )
>  {
>
> Fixes it ! :-)


Well, that's not a fix. This eliminates the case that something in
ARM-specific code (which I haven't tested) accidentally clears
_PGC_need_scrub.

OK, I think I know what the problem is. You are using
CONFIG_SEPARATE_XENHEAP, are you?


-boris

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 14/15] xen/x86: p2m-pod: Use typesafe gfn for the fields reclaim_single and max_guest

[Julien Grall]

Signed-off-by: Julien Grall 

---

Cc: George Dunlap 
Cc: Jan Beulich 
Cc: Andrew Cooper 
---
 xen/arch/x86/mm/p2m-pod.c | 11 +--
 xen/include/asm-x86/p2m.h |  4 ++--
 2 files changed, 7 insertions(+), 8 deletions(-)

diff --git a/xen/arch/x86/mm/p2m-pod.c b/xen/arch/x86/mm/p2m-pod.c
index 5c79444d7b..311762b1ce 100644
--- a/xen/arch/x86/mm/p2m-pod.c
+++ b/xen/arch/x86/mm/p2m-pod.c
@@ -977,10 +977,10 @@ p2m_pod_emergency_sweep(struct p2m_domain *p2m)
 p2m_type_t t;
 
 
-if ( p2m->pod.reclaim_single == 0 )
+if ( gfn_eq(p2m->pod.reclaim_single, _gfn(0)) )
 p2m->pod.reclaim_single = p2m->pod.max_guest;
 
-start = p2m->pod.reclaim_single;
+start = gfn_x(p2m->pod.reclaim_single);
 limit = (start > POD_SWEEP_LIMIT) ? (start - POD_SWEEP_LIMIT) : 0;
 
 /* FIXME: Figure out how to avoid superpages */
@@ -990,7 +990,7 @@ p2m_pod_emergency_sweep(struct p2m_domain *p2m)
  * careful about spinlock recursion limits and POD_SWEEP_STRIDE.
  */
 p2m_lock(p2m);
-for ( i = p2m->pod.reclaim_single; i > 0 ; i-- )
+for ( i = gfn_x(p2m->pod.reclaim_single); i > 0 ; i-- )
 {
 p2m_access_t a;
 (void)p2m->get_entry(p2m, _gfn(i), , , 0, NULL, NULL);
@@ -1020,7 +1020,7 @@ p2m_pod_emergency_sweep(struct p2m_domain *p2m)
 p2m_pod_zero_check(p2m, gfns, j);
 
 p2m_unlock(p2m);
-p2m->pod.reclaim_single = i ? i - 1 : i;
+p2m->pod.reclaim_single = _gfn(i ? i - 1 : i);
 
 }
 
@@ -1135,8 +1135,7 @@ p2m_pod_demand_populate(struct p2m_domain *p2m, gfn_t gfn,
 goto out_of_memory;
 
 /* Keep track of the highest gfn demand-populated by a guest fault */
-if ( gfn_x(gfn) > p2m->pod.max_guest )
-p2m->pod.max_guest = gfn_x(gfn);
+p2m->pod.max_guest = gfn_max(gfn, p2m->pod.max_guest);
 
 /*
  * Get a page f/ the cache.  A NULL return value indicates that the
diff --git a/xen/include/asm-x86/p2m.h b/xen/include/asm-x86/p2m.h
index 1ae9216404..e8a9dca480 100644
--- a/xen/include/asm-x86/p2m.h
+++ b/xen/include/asm-x86/p2m.h
@@ -316,8 +316,8 @@ struct p2m_domain {
  single;   /* Non-super lists   */
 long count,/* # of pages in cache lists */
  entry_count;  /* # of pages in p2m marked pod  */
-unsigned longreclaim_single; /* Last gpfn of a scan */
-unsigned longmax_guest;/* gpfn of max guest demand-populate */
+gfn_treclaim_single; /* Last gfn of a scan */
+gfn_tmax_guest;/* gfn of max guest demand-populate */
 
 /*
  * Tracking of the most recently populated PoD pages, for eager
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 09/15] xen/x86: p2m: Use typesafe GFN in p2m_set_entry

[Julien Grall]

Signed-off-by: Julien Grall 

---

Cc: George Dunlap 
Cc: Jan Beulich 
Cc: Andrew Cooper 
Cc: Tamas K Lengyel 
---
 xen/arch/x86/mm/hap/nested_hap.c |   2 +-
 xen/arch/x86/mm/mem_sharing.c|   3 +-
 xen/arch/x86/mm/p2m-pod.c|  36 +++--
 xen/arch/x86/mm/p2m.c| 112 ++-
 xen/include/asm-x86/p2m.h|   2 +-
 5 files changed, 85 insertions(+), 70 deletions(-)

diff --git a/xen/arch/x86/mm/hap/nested_hap.c b/xen/arch/x86/mm/hap/nested_hap.c
index 162afed46b..346fcb53e5 100644
--- a/xen/arch/x86/mm/hap/nested_hap.c
+++ b/xen/arch/x86/mm/hap/nested_hap.c
@@ -121,7 +121,7 @@ nestedhap_fix_p2m(struct vcpu *v, struct p2m_domain *p2m,
 gfn = (L2_gpa >> PAGE_SHIFT) & mask;
 mfn = _mfn((L0_gpa >> PAGE_SHIFT) & mask);
 
-rc = p2m_set_entry(p2m, gfn, mfn, page_order, p2mt, p2ma);
+rc = p2m_set_entry(p2m, _gfn(gfn), mfn, page_order, p2mt, p2ma);
 }
 
 p2m_unlock(p2m);
diff --git a/xen/arch/x86/mm/mem_sharing.c b/xen/arch/x86/mm/mem_sharing.c
index 4c1ace9b21..2f8c6fa746 100644
--- a/xen/arch/x86/mm/mem_sharing.c
+++ b/xen/arch/x86/mm/mem_sharing.c
@@ -1052,7 +1052,8 @@ int mem_sharing_add_to_physmap(struct domain *sd, 
unsigned long sgfn, shr_handle
 goto err_unlock;
 }
 
-ret = p2m_set_entry(p2m, cgfn, smfn, PAGE_ORDER_4K, p2m_ram_shared, a);
+ret = p2m_set_entry(p2m, _gfn(cgfn), smfn, PAGE_ORDER_4K,
+p2m_ram_shared, a);
 
 /* Tempted to turn this into an assert */
 if ( ret )
diff --git a/xen/arch/x86/mm/p2m-pod.c b/xen/arch/x86/mm/p2m-pod.c
index c8c8cff014..b8a51cf12a 100644
--- a/xen/arch/x86/mm/p2m-pod.c
+++ b/xen/arch/x86/mm/p2m-pod.c
@@ -565,7 +565,7 @@ p2m_pod_decrease_reservation(struct domain *d, gfn_t gfn, 
unsigned int order)
  * All PoD: Mark the whole region invalid and tell caller
  * we're done.
  */
-p2m_set_entry(p2m, gfn_x(gfn), INVALID_MFN, order, p2m_invalid,
+p2m_set_entry(p2m, gfn, INVALID_MFN, order, p2m_invalid,
   p2m->default_access);
 p2m->pod.entry_count -= 1UL << order;
 BUG_ON(p2m->pod.entry_count < 0);
@@ -609,7 +609,7 @@ p2m_pod_decrease_reservation(struct domain *d, gfn_t gfn, 
unsigned int order)
 n = 1UL << cur_order;
 if ( t == p2m_populate_on_demand )
 {
-p2m_set_entry(p2m, gfn_x(gfn) + i, INVALID_MFN, cur_order,
+p2m_set_entry(p2m, gfn_add(gfn, i), INVALID_MFN, cur_order,
   p2m_invalid, p2m->default_access);
 p2m->pod.entry_count -= n;
 BUG_ON(p2m->pod.entry_count < 0);
@@ -631,7 +631,7 @@ p2m_pod_decrease_reservation(struct domain *d, gfn_t gfn, 
unsigned int order)
 
 page = mfn_to_page(mfn);
 
-p2m_set_entry(p2m, gfn_x(gfn) + i, INVALID_MFN, cur_order,
+p2m_set_entry(p2m, gfn_add(gfn, i), INVALID_MFN, cur_order,
   p2m_invalid, p2m->default_access);
 p2m_tlb_flush_sync(p2m);
 for ( j = 0; j < n; ++j )
@@ -680,9 +680,10 @@ void p2m_pod_dump_data(struct domain *d)
  * in the p2m.
  */
 static int
-p2m_pod_zero_check_superpage(struct p2m_domain *p2m, unsigned long gfn)
+p2m_pod_zero_check_superpage(struct p2m_domain *p2m, unsigned long gfn_l)
 {
 mfn_t mfn, mfn0 = INVALID_MFN;
+gfn_t gfn = _gfn(gfn_l);
 p2m_type_t type, type0 = 0;
 unsigned long * map = NULL;
 int ret=0, reset = 0;
@@ -693,7 +694,7 @@ p2m_pod_zero_check_superpage(struct p2m_domain *p2m, 
unsigned long gfn)
 
 ASSERT(pod_locked_by_me(p2m));
 
-if ( !superpage_aligned(gfn) )
+if ( !superpage_aligned(gfn_l) )
 goto out;
 
 /* Allow an extra refcount for one shadow pt mapping in shadowed domains */
@@ -717,7 +718,7 @@ p2m_pod_zero_check_superpage(struct p2m_domain *p2m, 
unsigned long gfn)
 unsigned long k;
 const struct page_info *page;
 
-mfn = p2m->get_entry(p2m, _gfn(gfn +  i), , , 0,
+mfn = p2m->get_entry(p2m, gfn_add(gfn, i), , , 0,
  _order, NULL);
 
 /*
@@ -815,7 +816,7 @@ p2m_pod_zero_check_superpage(struct p2m_domain *p2m, 
unsigned long gfn)
 int d:16,order:16;
 } t;
 
-t.gfn = gfn;
+t.gfn = gfn_l;
 t.mfn = mfn_x(mfn);
 t.d = d->domain_id;
 t.order = 9;
@@ -898,7 +899,7 @@ p2m_pod_zero_check(struct p2m_domain *p2m, unsigned long 
*gfns, int count)
 }
 
 /* Try to remove the page, restoring old mapping if it fails. */
-p2m_set_entry(p2m, gfns[i], INVALID_MFN, PAGE_ORDER_4K,
+p2m_set_entry(p2m, _gfn(gfns[i]), INVALID_MFN, PAGE_ORDER_4K,
   p2m_populate_on_demand, p2m->default_access);
 
 /*
@@ -910,7 +911,7 @@ p2m_pod_zero_check(struct p2m_domain *p2m, 

[Xen-devel] [PATCH 13/15] xen/x86: p2m-pod: Use typesafe gfn in p2m_pod_demand_populate

[Julien Grall]

Signed-off-by: Julien Grall 

---

Cc: George Dunlap 
Cc: Jan Beulich 
Cc: Andrew Cooper 
---
 xen/arch/x86/mm/p2m-ept.c |  4 ++--
 xen/arch/x86/mm/p2m-pod.c | 12 ++--
 xen/arch/x86/mm/p2m-pt.c  |  6 +++---
 xen/include/asm-x86/p2m.h |  2 +-
 4 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/xen/arch/x86/mm/p2m-ept.c b/xen/arch/x86/mm/p2m-ept.c
index dff214cf7b..36627f1ce0 100644
--- a/xen/arch/x86/mm/p2m-ept.c
+++ b/xen/arch/x86/mm/p2m-ept.c
@@ -965,7 +965,7 @@ static mfn_t ept_get_entry(struct p2m_domain *p2m,
 index = gfn_remainder >> ( i * EPT_TABLE_ORDER);
 ept_entry = table + index;
 
-if ( !p2m_pod_demand_populate(p2m, gfn, i * EPT_TABLE_ORDER, q) )
+if ( !p2m_pod_demand_populate(p2m, gfn_t, i * EPT_TABLE_ORDER, q) )
 goto retry;
 else
 goto out;
@@ -987,7 +987,7 @@ static mfn_t ept_get_entry(struct p2m_domain *p2m,
 
 ASSERT(i == 0);
 
-if ( p2m_pod_demand_populate(p2m, gfn, 
+if ( p2m_pod_demand_populate(p2m, gfn_t,
 PAGE_ORDER_4K, q) )
 goto out;
 }
diff --git a/xen/arch/x86/mm/p2m-pod.c b/xen/arch/x86/mm/p2m-pod.c
index 0dd0f0a083..5c79444d7b 100644
--- a/xen/arch/x86/mm/p2m-pod.c
+++ b/xen/arch/x86/mm/p2m-pod.c
@@ -1076,13 +1076,13 @@ static void pod_eager_record(struct p2m_domain *p2m, 
gfn_t gfn,
 }
 
 int
-p2m_pod_demand_populate(struct p2m_domain *p2m, unsigned long gfn,
+p2m_pod_demand_populate(struct p2m_domain *p2m, gfn_t gfn,
 unsigned int order,
 p2m_query_t q)
 {
 struct domain *d = p2m->domain;
 struct page_info *p = NULL; /* Compiler warnings */
-gfn_t gfn_aligned = _gfn((gfn >> order) << order);
+gfn_t gfn_aligned = _gfn((gfn_x(gfn) >> order) << order);
 mfn_t mfn;
 unsigned long i;
 
@@ -1135,8 +1135,8 @@ p2m_pod_demand_populate(struct p2m_domain *p2m, unsigned 
long gfn,
 goto out_of_memory;
 
 /* Keep track of the highest gfn demand-populated by a guest fault */
-if ( gfn > p2m->pod.max_guest )
-p2m->pod.max_guest = gfn;
+if ( gfn_x(gfn) > p2m->pod.max_guest )
+p2m->pod.max_guest = gfn_x(gfn);
 
 /*
  * Get a page f/ the cache.  A NULL return value indicates that the
@@ -1170,7 +1170,7 @@ p2m_pod_demand_populate(struct p2m_domain *p2m, unsigned 
long gfn,
 int d:16,order:16;
 } t;
 
-t.gfn = gfn;
+t.gfn = gfn_x(gfn);
 t.mfn = mfn_x(mfn);
 t.d = d->domain_id;
 t.order = order;
@@ -1210,7 +1210,7 @@ remap_and_retry:
 int d:16;
 } t;
 
-t.gfn = gfn;
+t.gfn = gfn_x(gfn);
 t.d = d->domain_id;
 
 __trace_var(TRC_MEM_POD_SUPERPAGE_SPLINTER, 0, sizeof(t), );
diff --git a/xen/arch/x86/mm/p2m-pt.c b/xen/arch/x86/mm/p2m-pt.c
index 57878b1886..3dd4bef66e 100644
--- a/xen/arch/x86/mm/p2m-pt.c
+++ b/xen/arch/x86/mm/p2m-pt.c
@@ -802,7 +802,7 @@ pod_retry_l3:
 {
 if ( q & P2M_ALLOC )
 {
-if ( !p2m_pod_demand_populate(p2m, gfn, PAGE_ORDER_1G, q) )
+if ( !p2m_pod_demand_populate(p2m, gfn_t, PAGE_ORDER_1G, 
q) )
 goto pod_retry_l3;
 gdprintk(XENLOG_ERR, "%s: Allocate 1GB failed!\n", 
__func__);
 }
@@ -844,7 +844,7 @@ pod_retry_l2:
 if ( p2m_flags_to_type(flags) == p2m_populate_on_demand )
 {
 if ( q & P2M_ALLOC ) {
-if ( !p2m_pod_demand_populate(p2m, gfn, PAGE_ORDER_2M, q) )
+if ( !p2m_pod_demand_populate(p2m, gfn_t, PAGE_ORDER_2M, q) )
 goto pod_retry_l2;
 } else
 *t = p2m_populate_on_demand;
@@ -883,7 +883,7 @@ pod_retry_l1:
 if ( l1t == p2m_populate_on_demand )
 {
 if ( q & P2M_ALLOC ) {
-if ( !p2m_pod_demand_populate(p2m, gfn, PAGE_ORDER_4K, q) )
+if ( !p2m_pod_demand_populate(p2m, gfn_t, PAGE_ORDER_4K, q) )
 goto pod_retry_l1;
 } else
 *t = p2m_populate_on_demand;
diff --git a/xen/include/asm-x86/p2m.h b/xen/include/asm-x86/p2m.h
index 07ca02a173..1ae9216404 100644
--- a/xen/include/asm-x86/p2m.h
+++ b/xen/include/asm-x86/p2m.h
@@ -720,7 +720,7 @@ extern void audit_p2m(struct domain *d,
 
 /* Called by p2m code when demand-populating a PoD page */
 int
-p2m_pod_demand_populate(struct p2m_domain *p2m, unsigned long gfn,
+p2m_pod_demand_populate(struct p2m_domain *p2m, gfn_t gfn,
 unsigned int order,
 p2m_query_t q);
 
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 01/15] xen/x86: p2m-pod: Clean-up includes

[Julien Grall]

A lot of the headers are not necessary. At the same time, order them in the
alphabetical order.

Signed-off-by: Julien Grall 

---

Cc: George Dunlap 
Cc: Jan Beulich 
Cc: Andrew Cooper 
---
 xen/arch/x86/mm/p2m-pod.c | 11 +++
 1 file changed, 3 insertions(+), 8 deletions(-)

diff --git a/xen/arch/x86/mm/p2m-pod.c b/xen/arch/x86/mm/p2m-pod.c
index 4085b7f752..fec87e5224 100644
--- a/xen/arch/x86/mm/p2m-pod.c
+++ b/xen/arch/x86/mm/p2m-pod.c
@@ -19,18 +19,13 @@
  * along with this program; If not, see .
  */
 
-#include 
-#include 
 #include 
-#include 
-#include 
+#include 
+#include 
+#include 
 #include 
 #include 
 #include 
-#include  /* ept_p2m_init() */
-#include 
-#include 
-#include 
 
 #include "mm-locks.h"
 
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 04/15] xen/x86: p2m-pod: Fix coding style

[Julien Grall]

Also take the opportunity to:
- move from 1 << * to 1UL << *.
- use unsigned when possible
- move from unsigned int -> unsigned long for some induction
variables

Signed-off-by: Julien Grall 

---

Cc: George Dunlap 
Cc: Jan Beulich 
Cc: Andrew Cooper 
---
 xen/arch/x86/mm/p2m-pod.c | 102 +++---
 1 file changed, 52 insertions(+), 50 deletions(-)

diff --git a/xen/arch/x86/mm/p2m-pod.c b/xen/arch/x86/mm/p2m-pod.c
index 6f045081b5..f04d6e03e2 100644
--- a/xen/arch/x86/mm/p2m-pod.c
+++ b/xen/arch/x86/mm/p2m-pod.c
@@ -60,7 +60,7 @@ p2m_pod_cache_add(struct p2m_domain *p2m,
   struct page_info *page,
   unsigned int order)
 {
-int i;
+unsigned long i;
 struct page_info *p;
 struct domain *d = p2m->domain;
 
@@ -70,23 +70,24 @@ p2m_pod_cache_add(struct p2m_domain *p2m,
 mfn = page_to_mfn(page);
 
 /* Check to make sure this is a contiguous region */
-if( mfn_x(mfn) & ((1 << order) - 1) )
+if ( mfn_x(mfn) & ((1UL << order) - 1) )
 {
 printk("%s: mfn %lx not aligned order %u! (mask %lx)\n",
__func__, mfn_x(mfn), order, ((1UL << order) - 1));
 return -1;
 }
 
-for(i=0; i < 1 << order ; i++) {
+for ( i = 0; i < 1UL << order ; i++)
+{
 struct domain * od;
 
 p = mfn_to_page(_mfn(mfn_x(mfn) + i));
 od = page_get_owner(p);
-if(od != d)
+if ( od != d )
 {
 printk("%s: mfn %lx expected owner d%d, got owner d%d!\n",
__func__, mfn_x(mfn), d->domain_id,
-   od?od->domain_id:-1);
+   od ? od->domain_id : -1);
 return -1;
 }
 }
@@ -99,12 +100,12 @@ p2m_pod_cache_add(struct p2m_domain *p2m,
  * guaranteed to be zero; but by reclaiming zero pages, we implicitly
  * promise to provide zero pages. So we scrub pages before using.
  */
-for ( i = 0; i < (1 << order); i++ )
+for ( i = 0; i < (1UL << order); i++ )
 clear_domain_page(_mfn(mfn_x(page_to_mfn(page)) + i));
 
 /* First, take all pages off the domain list */
 lock_page_alloc(p2m);
-for(i=0; i < 1 << order ; i++)
+for ( i = 0; i < 1UL << order ; i++ )
 {
 p = page + i;
 page_list_del(p, >page_list);
@@ -128,7 +129,7 @@ p2m_pod_cache_add(struct p2m_domain *p2m,
 default:
 BUG();
 }
-p2m->pod.count += 1L << order;
+p2m->pod.count += 1UL << order;
 
 return 0;
 }
@@ -140,7 +141,7 @@ static struct page_info * p2m_pod_cache_get(struct 
p2m_domain *p2m,
 unsigned int order)
 {
 struct page_info *p = NULL;
-int i;
+unsigned long i;
 
 ASSERT(pod_locked_by_me(p2m));
 
@@ -162,7 +163,7 @@ static struct page_info * p2m_pod_cache_get(struct 
p2m_domain *p2m,
 p = page_list_remove_head(>pod.super);
 mfn = mfn_x(page_to_mfn(p));
 
-for ( i=0; ipod.single);
@@ -174,12 +175,12 @@ static struct page_info * p2m_pod_cache_get(struct 
p2m_domain *p2m,
 case PAGE_ORDER_2M:
 BUG_ON( page_list_empty(>pod.super) );
 p = page_list_remove_head(>pod.super);
-p2m->pod.count -= 1 << order;
+p2m->pod.count -= 1UL << order;
 break;
 case PAGE_ORDER_4K:
 BUG_ON( page_list_empty(>pod.single) );
 p = page_list_remove_head(>pod.single);
-p2m->pod.count -= 1;
+p2m->pod.count -= 1UL;
 break;
 default:
 BUG();
@@ -187,7 +188,7 @@ static struct page_info * p2m_pod_cache_get(struct 
p2m_domain *p2m,
 
 /* Put the pages back on the domain page_list */
 lock_page_alloc(p2m);
-for ( i = 0 ; i < (1 << order); i++ )
+for ( i = 0 ; i < (1UL << order); i++ )
 {
 BUG_ON(page_get_owner(p + i) != p2m->domain);
 page_list_add_tail(p + i, >domain->page_list);
@@ -251,7 +252,8 @@ p2m_pod_set_cache_target(struct p2m_domain *p2m, unsigned 
long pod_target, int p
 while ( pod_target < p2m->pod.count )
 {
 struct page_info * page;
-int order, i;
+unsigned int order;
+unsigned long i;
 
 if ( (p2m->pod.count - pod_target) > SUPERPAGE_PAGES
  && !page_list_empty(>pod.super) )
@@ -264,10 +266,10 @@ p2m_pod_set_cache_target(struct p2m_domain *p2m, unsigned 
long pod_target, int p
 ASSERT(page != NULL);
 
 /* Then free them */
-for ( i = 0 ; i < (1 << order) ; i++ )
+for ( i = 0 ; i < (1UL << order) ; i++ )
 {
 /* Copied from common/memory.c:guest_remove_page() */
-if ( unlikely(!get_page(page+i, d)) )
+if ( unlikely(!get_page(page + i, 

[Xen-devel] [PATCH 07/15] xen/x86: p2m-pod: Use typesafe gfn in p2m_pod_decrease_reservation

[Julien Grall]

Signed-off-by: Julien Grall 

---

Cc: George Dunlap 
Cc: Jan Beulich 
Cc: Andrew Cooper 
---
 xen/arch/arm/p2m.c   |  3 +--
 xen/arch/x86/mm/p2m-pod.c| 20 +---
 xen/common/memory.c  |  3 ++-
 xen/include/asm-arm/p2m.h| 13 -
 xen/include/asm-x86/p2m.h|  7 ---
 xen/include/xen/p2m-common.h | 13 +
 6 files changed, 25 insertions(+), 34 deletions(-)

diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
index c484469e6c..e321d78818 100644
--- a/xen/arch/arm/p2m.c
+++ b/xen/arch/arm/p2m.c
@@ -393,8 +393,7 @@ int guest_physmap_mark_populate_on_demand(struct domain *d,
 return -ENOSYS;
 }
 
-int p2m_pod_decrease_reservation(struct domain *d,
- xen_pfn_t gpfn,
+int p2m_pod_decrease_reservation(struct domain *d, gfn_t gfn,
  unsigned int order)
 {
 return -ENOSYS;
diff --git a/xen/arch/x86/mm/p2m-pod.c b/xen/arch/x86/mm/p2m-pod.c
index 34f5239b6d..eb74e5c01f 100644
--- a/xen/arch/x86/mm/p2m-pod.c
+++ b/xen/arch/x86/mm/p2m-pod.c
@@ -511,9 +511,7 @@ p2m_pod_zero_check_superpage(struct p2m_domain *p2m, 
unsigned long gfn);
  * allow decrease_reservation() to handle everything else.
  */
 int
-p2m_pod_decrease_reservation(struct domain *d,
- xen_pfn_t gpfn,
- unsigned int order)
+p2m_pod_decrease_reservation(struct domain *d, gfn_t gfn, unsigned int order)
 {
 int ret = 0;
 unsigned long i, n;
@@ -521,7 +519,7 @@ p2m_pod_decrease_reservation(struct domain *d,
 bool_t steal_for_cache;
 long pod, nonpod, ram;
 
-gfn_lock(p2m, gpfn, order);
+gfn_lock(p2m, gfn, order);
 pod_lock(p2m);
 
 /*
@@ -545,7 +543,7 @@ p2m_pod_decrease_reservation(struct domain *d,
 p2m_type_t t;
 unsigned int cur_order;
 
-p2m->get_entry(p2m, gpfn + i, , , 0, _order, NULL);
+p2m->get_entry(p2m, gfn_x(gfn) + i, , , 0, _order, NULL);
 n = 1UL << min(order, cur_order);
 if ( t == p2m_populate_on_demand )
 pod += n;
@@ -567,7 +565,7 @@ p2m_pod_decrease_reservation(struct domain *d,
  * All PoD: Mark the whole region invalid and tell caller
  * we're done.
  */
-p2m_set_entry(p2m, gpfn, INVALID_MFN, order, p2m_invalid,
+p2m_set_entry(p2m, gfn_x(gfn), INVALID_MFN, order, p2m_invalid,
   p2m->default_access);
 p2m->pod.entry_count -= 1UL << order;
 BUG_ON(p2m->pod.entry_count < 0);
@@ -584,7 +582,7 @@ p2m_pod_decrease_reservation(struct domain *d,
  * - not all of the pages were RAM (now knowing order < SUPERPAGE_ORDER)
  */
 if ( steal_for_cache && order < SUPERPAGE_ORDER && ram == (1UL << order) &&
- p2m_pod_zero_check_superpage(p2m, gpfn & ~(SUPERPAGE_PAGES - 1)) )
+ p2m_pod_zero_check_superpage(p2m, gfn_x(gfn) & ~(SUPERPAGE_PAGES - 
1)) )
 {
 pod = 1UL << order;
 ram = nonpod = 0;
@@ -605,13 +603,13 @@ p2m_pod_decrease_reservation(struct domain *d,
 p2m_access_t a;
 unsigned int cur_order;
 
-mfn = p2m->get_entry(p2m, gpfn + i, , , 0, _order, NULL);
+mfn = p2m->get_entry(p2m, gfn_x(gfn) + i, , , 0, _order, NULL);
 if ( order < cur_order )
 cur_order = order;
 n = 1UL << cur_order;
 if ( t == p2m_populate_on_demand )
 {
-p2m_set_entry(p2m, gpfn + i, INVALID_MFN, cur_order,
+p2m_set_entry(p2m, gfn_x(gfn) + i, INVALID_MFN, cur_order,
   p2m_invalid, p2m->default_access);
 p2m->pod.entry_count -= n;
 BUG_ON(p2m->pod.entry_count < 0);
@@ -633,7 +631,7 @@ p2m_pod_decrease_reservation(struct domain *d,
 
 page = mfn_to_page(mfn);
 
-p2m_set_entry(p2m, gpfn + i, INVALID_MFN, cur_order,
+p2m_set_entry(p2m, gfn_x(gfn) + i, INVALID_MFN, cur_order,
   p2m_invalid, p2m->default_access);
 p2m_tlb_flush_sync(p2m);
 for ( j = 0; j < n; ++j )
@@ -663,7 +661,7 @@ out_entry_check:
 
 out_unlock:
 pod_unlock(p2m);
-gfn_unlock(p2m, gpfn, order);
+gfn_unlock(p2m, gfn, order);
 return ret;
 }
 
diff --git a/xen/common/memory.c b/xen/common/memory.c
index a2abf554e3..ad987e0f29 100644
--- a/xen/common/memory.c
+++ b/xen/common/memory.c
@@ -417,7 +417,8 @@ static void decrease_reservation(struct memop_args *a)
 
 /* See if populate-on-demand wants to handle this */
 if ( is_hvm_domain(a->domain)
- && p2m_pod_decrease_reservation(a->domain, gmfn, a->extent_order) 
)
+ && p2m_pod_decrease_reservation(a->domain, _gfn(gmfn),
+ a->extent_order) )
 continue;
 
 for ( j = 0; j < (1 << a->extent_order); j++ )
diff --git 

[Xen-devel] [PATCH 08/15] xen/x86: p2m: Use typesafe gfn for the P2M callbacks get_entry and set_entry

[Julien Grall]

Signed-off-by: Julien Grall 

Cc: Jan Beulich 
Cc: Andrew Cooper 
Cc: Razvan Cojocaru 
Cc: Tamas K Lengyel 
Cc: George Dunlap 
Cc: Jun Nakajima 
Cc: Kevin Tian 
---
 xen/arch/x86/hvm/hvm.c|  2 +-
 xen/arch/x86/mm/mem_access.c  | 19 +--
 xen/arch/x86/mm/mem_sharing.c |  4 +--
 xen/arch/x86/mm/p2m-ept.c |  6 ++--
 xen/arch/x86/mm/p2m-pod.c | 15 +
 xen/arch/x86/mm/p2m-pt.c  |  6 ++--
 xen/arch/x86/mm/p2m.c | 77 +++
 xen/include/asm-x86/p2m.h |  4 +--
 8 files changed, 73 insertions(+), 60 deletions(-)

diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 6cb903def5..53be8c093c 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -1787,7 +1787,7 @@ int hvm_hap_nested_page_fault(paddr_t gpa, unsigned long 
gla,
 {
 bool_t sve;
 
-p2m->get_entry(p2m, gfn, , , 0, NULL, );
+p2m->get_entry(p2m, _gfn(gfn), , , 0, NULL, );
 
 if ( !sve && altp2m_vcpu_emulate_ve(curr) )
 {
diff --git a/xen/arch/x86/mm/mem_access.c b/xen/arch/x86/mm/mem_access.c
index 9211fc0abe..948e377e69 100644
--- a/xen/arch/x86/mm/mem_access.c
+++ b/xen/arch/x86/mm/mem_access.c
@@ -66,7 +66,7 @@ static int _p2m_get_mem_access(struct p2m_domain *p2m, gfn_t 
gfn,
 }
 
 gfn_lock(p2m, gfn, 0);
-mfn = p2m->get_entry(p2m, gfn_x(gfn), , , 0, NULL, NULL);
+mfn = p2m->get_entry(p2m, gfn, , , 0, NULL, NULL);
 gfn_unlock(p2m, gfn, 0);
 
 if ( mfn_eq(mfn, INVALID_MFN) )
@@ -142,7 +142,7 @@ bool p2m_mem_access_check(paddr_t gpa, unsigned long gla,
   vm_event_request_t **req_ptr)
 {
 struct vcpu *v = current;
-unsigned long gfn = gpa >> PAGE_SHIFT;
+gfn_t gfn = gaddr_to_gfn(gpa);
 struct domain *d = v->domain;
 struct p2m_domain *p2m = NULL;
 mfn_t mfn;
@@ -215,7 +215,7 @@ bool p2m_mem_access_check(paddr_t gpa, unsigned long gla,
 *req_ptr = req;
 
 req->reason = VM_EVENT_REASON_MEM_ACCESS;
-req->u.mem_access.gfn = gfn;
+req->u.mem_access.gfn = gfn_x(gfn);
 req->u.mem_access.offset = gpa & ((1 << PAGE_SHIFT) - 1);
 if ( npfec.gla_valid )
 {
@@ -247,7 +247,7 @@ int p2m_set_altp2m_mem_access(struct domain *d, struct 
p2m_domain *hp2m,
 unsigned long gfn_l = gfn_x(gfn);
 int rc;
 
-mfn = ap2m->get_entry(ap2m, gfn_l, , _a, 0, NULL, NULL);
+mfn = ap2m->get_entry(ap2m, gfn, , _a, 0, NULL, NULL);
 
 /* Check host p2m if no valid entry in alternate */
 if ( !mfn_valid(mfn) )
@@ -264,16 +264,16 @@ int p2m_set_altp2m_mem_access(struct domain *d, struct 
p2m_domain *hp2m,
 if ( page_order != PAGE_ORDER_4K )
 {
 unsigned long mask = ~((1UL << page_order) - 1);
-unsigned long gfn2_l = gfn_l & mask;
+gfn_t gfn2 = _gfn(gfn_l & mask);
 mfn_t mfn2 = _mfn(mfn_x(mfn) & mask);
 
-rc = ap2m->set_entry(ap2m, gfn2_l, mfn2, page_order, t, old_a, 1);
+rc = ap2m->set_entry(ap2m, gfn2, mfn2, page_order, t, old_a, 1);
 if ( rc )
 return rc;
 }
 }
 
-return ap2m->set_entry(ap2m, gfn_l, mfn, PAGE_ORDER_4K, t, a,
+return ap2m->set_entry(ap2m, gfn, mfn, PAGE_ORDER_4K, t, a,
  (current->domain != d));
 }
 
@@ -295,10 +295,9 @@ static int set_mem_access(struct domain *d, struct 
p2m_domain *p2m,
 mfn_t mfn;
 p2m_access_t _a;
 p2m_type_t t;
-unsigned long gfn_l = gfn_x(gfn);
 
-mfn = p2m->get_entry(p2m, gfn_l, , &_a, 0, NULL, NULL);
-rc = p2m->set_entry(p2m, gfn_l, mfn, PAGE_ORDER_4K, t, a, -1);
+mfn = p2m->get_entry(p2m, gfn, , &_a, 0, NULL, NULL);
+rc = p2m->set_entry(p2m, gfn, mfn, PAGE_ORDER_4K, t, a, -1);
 }
 
 return rc;
diff --git a/xen/arch/x86/mm/mem_sharing.c b/xen/arch/x86/mm/mem_sharing.c
index 12fb9cc51f..4c1ace9b21 100644
--- a/xen/arch/x86/mm/mem_sharing.c
+++ b/xen/arch/x86/mm/mem_sharing.c
@@ -1234,7 +1234,7 @@ int relinquish_shared_pages(struct domain *d)
 
 if ( atomic_read(>shr_pages) == 0 )
 break;
-mfn = p2m->get_entry(p2m, gfn, , , 0, NULL, NULL);
+mfn = p2m->get_entry(p2m, _gfn(gfn), , , 0, NULL, NULL);
 if ( mfn_valid(mfn) && (t == p2m_ram_shared) )
 {
 /* Does not fail with ENOMEM given the DESTROY flag */
@@ -1243,7 +1243,7 @@ int relinquish_shared_pages(struct domain *d)
 /* Clear out the p2m entry so no one else may try to
  * unshare.  Must succeed: we just read the old entry and
  * we hold the p2m lock. */
-set_rc = p2m->set_entry(p2m, gfn, _mfn(0), PAGE_ORDER_4K,
+set_rc 

[Xen-devel] [PATCH 03/15] xen/x86: p2m-pod: Fix coding style for comments

[Julien Grall]

Signed-off-by: Julien Grall 

---

Cc: George Dunlap 
Cc: Jan Beulich 
Cc: Andrew Cooper 
---
 xen/arch/x86/mm/p2m-pod.c | 154 ++
 1 file changed, 102 insertions(+), 52 deletions(-)

diff --git a/xen/arch/x86/mm/p2m-pod.c b/xen/arch/x86/mm/p2m-pod.c
index 1f07441259..6f045081b5 100644
--- a/xen/arch/x86/mm/p2m-pod.c
+++ b/xen/arch/x86/mm/p2m-pod.c
@@ -155,8 +155,10 @@ static struct page_info * p2m_pod_cache_get(struct 
p2m_domain *p2m,
 
 BUG_ON( page_list_empty(>pod.super) );
 
-/* Break up a superpage to make single pages. NB count doesn't
- * need to be adjusted. */
+/*
+ * Break up a superpage to make single pages. NB count doesn't
+ * need to be adjusted.
+ */
 p = page_list_remove_head(>pod.super);
 mfn = mfn_x(page_to_mfn(p));
 
@@ -242,8 +244,10 @@ p2m_pod_set_cache_target(struct p2m_domain *p2m, unsigned 
long pod_target, int p
 }
 
 /* Decreasing the target */
-/* We hold the pod lock here, so we don't need to worry about
- * cache disappearing under our feet. */
+/*
+ * We hold the pod lock here, so we don't need to worry about
+ * cache disappearing under our feet.
+ */
 while ( pod_target < p2m->pod.count )
 {
 struct page_info * page;
@@ -345,15 +349,19 @@ p2m_pod_set_mem_target(struct domain *d, unsigned long 
target)
 if ( d->is_dying )
 goto out;
 
-/* T' < B: Don't reduce the cache size; let the balloon driver
- * take care of it. */
+/*
+ * T' < B: Don't reduce the cache size; let the balloon driver
+ * take care of it.
+ */
 if ( target < d->tot_pages )
 goto out;
 
 pod_target = target - populated;
 
-/* B < T': Set the cache size equal to # of outstanding entries,
- * let the balloon driver fill in the rest. */
+/*
+ * B < T': Set the cache size equal to # of outstanding entries,
+ * let the balloon driver fill in the rest.
+ */
 if ( populated > 0 && pod_target > p2m->pod.entry_count )
 pod_target = p2m->pod.entry_count;
 
@@ -491,7 +499,8 @@ static int
 p2m_pod_zero_check_superpage(struct p2m_domain *p2m, unsigned long gfn);
 
 
-/* This function is needed for two reasons:
+/*
+ * This function is needed for two reasons:
  * + To properly handle clearing of PoD entries
  * + To "steal back" memory being freed for the PoD cache, rather than
  *   releasing it.
@@ -513,8 +522,10 @@ p2m_pod_decrease_reservation(struct domain *d,
 gfn_lock(p2m, gpfn, order);
 pod_lock(p2m);
 
-/* If we don't have any outstanding PoD entries, let things take their
- * course */
+/*
+ * If we don't have any outstanding PoD entries, let things take their
+ * course.
+ */
 if ( p2m->pod.entry_count == 0 )
 goto out_unlock;
 
@@ -550,8 +561,10 @@ p2m_pod_decrease_reservation(struct domain *d,
 
 if ( !nonpod )
 {
-/* All PoD: Mark the whole region invalid and tell caller
- * we're done. */
+/*
+ * All PoD: Mark the whole region invalid and tell caller
+ * we're done.
+ */
 p2m_set_entry(p2m, gpfn, INVALID_MFN, order, p2m_invalid,
   p2m->default_access);
 p2m->pod.entry_count-=(1<= SUPERPAGE_ORDER (the loop below will take care of this)
  * - not all of the pages were RAM (now knowing order < SUPERPAGE_ORDER)
  */
-if ( steal_for_cache && order < SUPERPAGE_ORDER && ram == (1 << order) &&
+if ( steal_for_cache && order < SUPERPAGE_ORDER && ram == (1UL << order) &&
  p2m_pod_zero_check_superpage(p2m, gpfn & ~(SUPERPAGE_PAGES - 1)) )
 {
-pod = 1 << order;
+pod = 1UL << order;
 ram = nonpod = 0;
 ASSERT(steal_for_cache == (p2m->pod.entry_count > p2m->pod.count));
 }
 
-/* Process as long as:
+/*
+ * Process as long as:
  * + There are PoD entries to handle, or
  * + There is ram left, and we want to steal it
  */
@@ -631,8 +645,10 @@ p2m_pod_decrease_reservation(struct domain *d,
 }
 }
 
-/* If there are no more non-PoD entries, tell decrease_reservation() that
- * there's nothing left to do. */
+/*
+ * If there are no more non-PoD entries, tell decrease_reservation() that
+ * there's nothing left to do.
+ */
 if ( nonpod == 0 )
 ret = 1;
 
@@ -658,9 +674,11 @@ void p2m_pod_dump_data(struct domain *d)
 }
 
 
-/* Search for all-zero superpages to be reclaimed as superpages for the
+/*
+ * Search for all-zero superpages to be reclaimed as superpages for the
  * PoD cache. Must be called w/ pod lock held, must lock the superpage
- * in the p2m */
+ * in the p2m.
+ */
 static int
 

[Xen-devel] [PATCH 11/15] xen/x86: p2m-pod: Clean-up p2m_pod_zero_check

[Julien Grall]

Signed-off-by: Julien Grall 

---

Cc: George Dunlap 
Cc: Jan Beulich 
Cc: Andrew Cooper 
---
 xen/arch/x86/mm/p2m-pod.c | 10 ++
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/xen/arch/x86/mm/p2m-pod.c b/xen/arch/x86/mm/p2m-pod.c
index 176d06cb42..611a087855 100644
--- a/xen/arch/x86/mm/p2m-pod.c
+++ b/xen/arch/x86/mm/p2m-pod.c
@@ -861,17 +861,19 @@ p2m_pod_zero_check(struct p2m_domain *p2m, unsigned long 
*gfns, int count)
 for ( i = 0; i < count; i++ )
 {
 p2m_access_t a;
+struct page_info *pg;
 
 mfns[i] = p2m->get_entry(p2m, _gfn(gfns[i]), types + i, ,
  0, NULL, NULL);
+pg = mfn_to_page(mfns[i]);
+
 /*
  * If this is ram, and not a pagetable or from the xen heap, and
  * probably not mapped elsewhere, map it; otherwise, skip.
  */
-if ( p2m_is_ram(types[i])
- && ( (mfn_to_page(mfns[i])->count_info & PGC_allocated) != 0 )
- && ( (mfn_to_page(mfns[i])->count_info & 
(PGC_page_table|PGC_xen_heap)) == 0 )
- && ( (mfn_to_page(mfns[i])->count_info & PGC_count_mask) <= 
max_ref ) )
+if ( p2m_is_ram(types[i]) && (pg->count_info & PGC_allocated) &&
+ ((pg->count_info & (PGC_page_table | PGC_xen_heap)) == 0) &&
+ ((pg->count_info & (PGC_count_mask)) <= max_ref) )
 map[i] = map_domain_page(mfns[i]);
 else
 map[i] = NULL;
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 15/15] xen/x86: p2m-pod: Rework prototype of p2m_pod_demand_populate

[Julien Grall]

- Switch the return type to bool
- Remove the parameter p2m_query_t q as it is not used

Signed-off-by: Julien Grall 

---

Cc: George Dunlap 
Cc: Jan Beulich 
Cc: Andrew Cooper 
---
 xen/arch/x86/mm/p2m-ept.c |  5 ++---
 xen/arch/x86/mm/p2m-pod.c | 15 +++
 xen/arch/x86/mm/p2m-pt.c  |  6 +++---
 xen/include/asm-x86/p2m.h |  6 ++
 4 files changed, 14 insertions(+), 18 deletions(-)

diff --git a/xen/arch/x86/mm/p2m-ept.c b/xen/arch/x86/mm/p2m-ept.c
index 36627f1ce0..641b90ec07 100644
--- a/xen/arch/x86/mm/p2m-ept.c
+++ b/xen/arch/x86/mm/p2m-ept.c
@@ -965,7 +965,7 @@ static mfn_t ept_get_entry(struct p2m_domain *p2m,
 index = gfn_remainder >> ( i * EPT_TABLE_ORDER);
 ept_entry = table + index;
 
-if ( !p2m_pod_demand_populate(p2m, gfn_t, i * EPT_TABLE_ORDER, q) )
+if ( p2m_pod_demand_populate(p2m, gfn_t, i * EPT_TABLE_ORDER) )
 goto retry;
 else
 goto out;
@@ -987,8 +987,7 @@ static mfn_t ept_get_entry(struct p2m_domain *p2m,
 
 ASSERT(i == 0);
 
-if ( p2m_pod_demand_populate(p2m, gfn_t,
-PAGE_ORDER_4K, q) )
+if ( !p2m_pod_demand_populate(p2m, gfn_t, PAGE_ORDER_4K) )
 goto out;
 }
 
diff --git a/xen/arch/x86/mm/p2m-pod.c b/xen/arch/x86/mm/p2m-pod.c
index 311762b1ce..89b7dc949d 100644
--- a/xen/arch/x86/mm/p2m-pod.c
+++ b/xen/arch/x86/mm/p2m-pod.c
@@ -1075,10 +1075,9 @@ static void pod_eager_record(struct p2m_domain *p2m, 
gfn_t gfn,
 mrp->idx %= ARRAY_SIZE(mrp->list);
 }
 
-int
+bool
 p2m_pod_demand_populate(struct p2m_domain *p2m, gfn_t gfn,
-unsigned int order,
-p2m_query_t q)
+unsigned int order)
 {
 struct domain *d = p2m->domain;
 struct page_info *p = NULL; /* Compiler warnings */
@@ -1116,7 +1115,7 @@ p2m_pod_demand_populate(struct p2m_domain *p2m, gfn_t gfn,
  */
 p2m_set_entry(p2m, gfn_aligned, INVALID_MFN, PAGE_ORDER_2M,
   p2m_populate_on_demand, p2m->default_access);
-return 0;
+return true;
 }
 
 /* Only reclaim if we're in actual need of more cache. */
@@ -1178,7 +1177,7 @@ p2m_pod_demand_populate(struct p2m_domain *p2m, gfn_t gfn,
 }
 
 pod_unlock(p2m);
-return 0;
+return true;
 out_of_memory:
 pod_unlock(p2m);
 
@@ -1186,10 +1185,10 @@ out_of_memory:
__func__, d->domain_id, d->tot_pages, p2m->pod.entry_count,
current->domain->domain_id);
 domain_crash(d);
-return -1;
+return false;
 out_fail:
 pod_unlock(p2m);
-return -1;
+return false;
 remap_and_retry:
 BUG_ON(order != PAGE_ORDER_2M);
 pod_unlock(p2m);
@@ -1215,7 +1214,7 @@ remap_and_retry:
 __trace_var(TRC_MEM_POD_SUPERPAGE_SPLINTER, 0, sizeof(t), );
 }
 
-return 0;
+return true;
 }
 
 
diff --git a/xen/arch/x86/mm/p2m-pt.c b/xen/arch/x86/mm/p2m-pt.c
index 3dd4bef66e..86946a09db 100644
--- a/xen/arch/x86/mm/p2m-pt.c
+++ b/xen/arch/x86/mm/p2m-pt.c
@@ -802,7 +802,7 @@ pod_retry_l3:
 {
 if ( q & P2M_ALLOC )
 {
-if ( !p2m_pod_demand_populate(p2m, gfn_t, PAGE_ORDER_1G, 
q) )
+if ( p2m_pod_demand_populate(p2m, gfn_t, PAGE_ORDER_1G) )
 goto pod_retry_l3;
 gdprintk(XENLOG_ERR, "%s: Allocate 1GB failed!\n", 
__func__);
 }
@@ -844,7 +844,7 @@ pod_retry_l2:
 if ( p2m_flags_to_type(flags) == p2m_populate_on_demand )
 {
 if ( q & P2M_ALLOC ) {
-if ( !p2m_pod_demand_populate(p2m, gfn_t, PAGE_ORDER_2M, q) )
+if ( p2m_pod_demand_populate(p2m, gfn_t, PAGE_ORDER_2M) )
 goto pod_retry_l2;
 } else
 *t = p2m_populate_on_demand;
@@ -883,7 +883,7 @@ pod_retry_l1:
 if ( l1t == p2m_populate_on_demand )
 {
 if ( q & P2M_ALLOC ) {
-if ( !p2m_pod_demand_populate(p2m, gfn_t, PAGE_ORDER_4K, q) )
+if ( p2m_pod_demand_populate(p2m, gfn_t, PAGE_ORDER_4K) )
 goto pod_retry_l1;
 } else
 *t = p2m_populate_on_demand;
diff --git a/xen/include/asm-x86/p2m.h b/xen/include/asm-x86/p2m.h
index e8a9dca480..70f00c332f 100644
--- a/xen/include/asm-x86/p2m.h
+++ b/xen/include/asm-x86/p2m.h
@@ -719,10 +719,8 @@ extern void audit_p2m(struct domain *d,
 #endif
 
 /* Called by p2m code when demand-populating a PoD page */
-int
-p2m_pod_demand_populate(struct p2m_domain *p2m, gfn_t gfn,
-unsigned int order,
-p2m_query_t q);
+bool
+p2m_pod_demand_populate(struct p2m_domain *p2m, gfn_t gfn, unsigned int order);
 
 /*
  * Functions specific to the p2m-pt 

[Xen-devel] [PATCH 10/15] xen/x86: p2m-pod: Use typesafe GFN in pod_eager_record

[Julien Grall]

Signed-off-by: Julien Grall 

---

Cc: George Dunlap 
Cc: Jan Beulich 
Cc: Andrew Cooper 
---
 xen/arch/x86/mm/p2m-pod.c | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/xen/arch/x86/mm/p2m-pod.c b/xen/arch/x86/mm/p2m-pod.c
index b8a51cf12a..176d06cb42 100644
--- a/xen/arch/x86/mm/p2m-pod.c
+++ b/xen/arch/x86/mm/p2m-pod.c
@@ -1062,15 +1062,15 @@ static void pod_eager_reclaim(struct p2m_domain *p2m)
 } while ( (p2m->pod.count == 0) && (i < ARRAY_SIZE(mrp->list)) );
 }
 
-static void pod_eager_record(struct p2m_domain *p2m,
- unsigned long gfn, unsigned int order)
+static void pod_eager_record(struct p2m_domain *p2m, gfn_t gfn,
+ unsigned int order)
 {
 struct pod_mrp_list *mrp = >pod.mrp;
 
-ASSERT(gfn != gfn_x(INVALID_GFN));
+ASSERT(!gfn_eq(gfn, INVALID_GFN));
 
 mrp->list[mrp->idx++] =
-gfn | (order == PAGE_ORDER_2M ? POD_LAST_SUPERPAGE : 0);
+gfn_x(gfn) | (order == PAGE_ORDER_2M ? POD_LAST_SUPERPAGE : 0);
 mrp->idx %= ARRAY_SIZE(mrp->list);
 }
 
@@ -1160,7 +1160,7 @@ p2m_pod_demand_populate(struct p2m_domain *p2m, unsigned 
long gfn,
 p2m->pod.entry_count -= (1UL << order);
 BUG_ON(p2m->pod.entry_count < 0);
 
-pod_eager_record(p2m, gfn_x(gfn_aligned), order);
+pod_eager_record(p2m, gfn_aligned, order);
 
 if ( tb_init_done )
 {
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 06/15] xen/x86: p2m-pod: Clean-up use of typesafe MFN

[Julien Grall]

Some unboxing/boxing can be avoided by using mfn_add(...) instead.

Signed-off-by: Julien Grall 

---

Cc: George Dunlap 
Cc: Jan Beulich 
Cc: Andrew Cooper 
---
 xen/arch/x86/mm/p2m-pod.c | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/xen/arch/x86/mm/p2m-pod.c b/xen/arch/x86/mm/p2m-pod.c
index bcc87aee03..34f5239b6d 100644
--- a/xen/arch/x86/mm/p2m-pod.c
+++ b/xen/arch/x86/mm/p2m-pod.c
@@ -101,7 +101,7 @@ p2m_pod_cache_add(struct p2m_domain *p2m,
  * promise to provide zero pages. So we scrub pages before using.
  */
 for ( i = 0; i < (1UL << order); i++ )
-clear_domain_page(_mfn(mfn_x(page_to_mfn(page)) + i));
+clear_domain_page(mfn_add(page_to_mfn(page), i));
 
 /* First, take all pages off the domain list */
 lock_page_alloc(p2m);
@@ -743,7 +743,7 @@ p2m_pod_zero_check_superpage(struct p2m_domain *p2m, 
unsigned long gfn)
 mfn0 = mfn;
 type0 = type;
 }
-else if ( type != type0 || mfn_x(mfn) != (mfn_x(mfn0) + i) )
+else if ( type != type0 || !mfn_eq(mfn, mfn_add(mfn0, i)) )
 goto out;
 
 n = 1UL << min(cur_order, SUPERPAGE_ORDER + 0U);
@@ -758,7 +758,7 @@ p2m_pod_zero_check_superpage(struct p2m_domain *p2m, 
unsigned long gfn)
 for ( i = 0; i < SUPERPAGE_PAGES; i++ )
 {
 /* Quick zero-check */
-map = map_domain_page(_mfn(mfn_x(mfn0) + i));
+map = map_domain_page(mfn_add(mfn0, i));
 
 for ( j = 0; j < 16; j++ )
 if ( *(map + j) != 0 )
@@ -783,7 +783,7 @@ p2m_pod_zero_check_superpage(struct p2m_domain *p2m, 
unsigned long gfn)
  */
 for ( i = 0; i < SUPERPAGE_PAGES; i++ )
 {
-mfn = _mfn(mfn_x(mfn0) + i);
+mfn = mfn_add(mfn0, i);
 if ( (mfn_to_page(mfn)->count_info & PGC_count_mask) > 1 )
 {
 reset = 1;
@@ -794,7 +794,7 @@ p2m_pod_zero_check_superpage(struct p2m_domain *p2m, 
unsigned long gfn)
 /* Finally, do a full zero-check */
 for ( i = 0; i < SUPERPAGE_PAGES; i++ )
 {
-map = map_domain_page(_mfn(mfn_x(mfn0) + i));
+map = map_domain_page(mfn_add(mfn0, i));
 
 for ( j = 0; j < (PAGE_SIZE / sizeof(*map)); j++ )
 if ( *(map+j) != 0 )
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 12/15] xen/x86: p2m-pod: Use typesafe gfn in p2m_pod_zero_check

[Julien Grall]

At the same time make the array gfns const has it is not modified within
the function.

Signed-off-by: Julien Grall 

---

Cc: George Dunlap 
Cc: Jan Beulich 
Cc: Andrew Cooper 
---
 xen/arch/x86/mm/p2m-pod.c | 37 ++---
 1 file changed, 18 insertions(+), 19 deletions(-)

diff --git a/xen/arch/x86/mm/p2m-pod.c b/xen/arch/x86/mm/p2m-pod.c
index 611a087855..0dd0f0a083 100644
--- a/xen/arch/x86/mm/p2m-pod.c
+++ b/xen/arch/x86/mm/p2m-pod.c
@@ -498,7 +498,7 @@ p2m_pod_offline_or_broken_replace(struct page_info *p)
 }
 
 static int
-p2m_pod_zero_check_superpage(struct p2m_domain *p2m, unsigned long gfn);
+p2m_pod_zero_check_superpage(struct p2m_domain *p2m, gfn_t gfn);
 
 
 /*
@@ -582,7 +582,7 @@ p2m_pod_decrease_reservation(struct domain *d, gfn_t gfn, 
unsigned int order)
  * - not all of the pages were RAM (now knowing order < SUPERPAGE_ORDER)
  */
 if ( steal_for_cache && order < SUPERPAGE_ORDER && ram == (1UL << order) &&
- p2m_pod_zero_check_superpage(p2m, gfn_x(gfn) & ~(SUPERPAGE_PAGES - 
1)) )
+ p2m_pod_zero_check_superpage(p2m, _gfn(gfn_x(gfn) & ~(SUPERPAGE_PAGES 
- 1))) )
 {
 pod = 1UL << order;
 ram = nonpod = 0;
@@ -680,10 +680,9 @@ void p2m_pod_dump_data(struct domain *d)
  * in the p2m.
  */
 static int
-p2m_pod_zero_check_superpage(struct p2m_domain *p2m, unsigned long gfn_l)
+p2m_pod_zero_check_superpage(struct p2m_domain *p2m, gfn_t gfn)
 {
 mfn_t mfn, mfn0 = INVALID_MFN;
-gfn_t gfn = _gfn(gfn_l);
 p2m_type_t type, type0 = 0;
 unsigned long * map = NULL;
 int ret=0, reset = 0;
@@ -694,7 +693,7 @@ p2m_pod_zero_check_superpage(struct p2m_domain *p2m, 
unsigned long gfn_l)
 
 ASSERT(pod_locked_by_me(p2m));
 
-if ( !superpage_aligned(gfn_l) )
+if ( !superpage_aligned(gfn_x(gfn)) )
 goto out;
 
 /* Allow an extra refcount for one shadow pt mapping in shadowed domains */
@@ -816,7 +815,7 @@ p2m_pod_zero_check_superpage(struct p2m_domain *p2m, 
unsigned long gfn_l)
 int d:16,order:16;
 } t;
 
-t.gfn = gfn_l;
+t.gfn = gfn_x(gfn);
 t.mfn = mfn_x(mfn);
 t.d = d->domain_id;
 t.order = 9;
@@ -843,7 +842,7 @@ out:
 }
 
 static void
-p2m_pod_zero_check(struct p2m_domain *p2m, unsigned long *gfns, int count)
+p2m_pod_zero_check(struct p2m_domain *p2m, const gfn_t *gfns, int count)
 {
 mfn_t mfns[count];
 p2m_type_t types[count];
@@ -863,7 +862,7 @@ p2m_pod_zero_check(struct p2m_domain *p2m, unsigned long 
*gfns, int count)
 p2m_access_t a;
 struct page_info *pg;
 
-mfns[i] = p2m->get_entry(p2m, _gfn(gfns[i]), types + i, ,
+mfns[i] = p2m->get_entry(p2m, gfns[i], types + i, ,
  0, NULL, NULL);
 pg = mfn_to_page(mfns[i]);
 
@@ -901,7 +900,7 @@ p2m_pod_zero_check(struct p2m_domain *p2m, unsigned long 
*gfns, int count)
 }
 
 /* Try to remove the page, restoring old mapping if it fails. */
-p2m_set_entry(p2m, _gfn(gfns[i]), INVALID_MFN, PAGE_ORDER_4K,
+p2m_set_entry(p2m, gfns[i], INVALID_MFN, PAGE_ORDER_4K,
   p2m_populate_on_demand, p2m->default_access);
 
 /*
@@ -913,7 +912,7 @@ p2m_pod_zero_check(struct p2m_domain *p2m, unsigned long 
*gfns, int count)
 unmap_domain_page(map[i]);
 map[i] = NULL;
 
-p2m_set_entry(p2m, _gfn(gfns[i]), mfns[i], PAGE_ORDER_4K,
+p2m_set_entry(p2m, gfns[i], mfns[i], PAGE_ORDER_4K,
 types[i], p2m->default_access);
 
 continue;
@@ -940,7 +939,7 @@ p2m_pod_zero_check(struct p2m_domain *p2m, unsigned long 
*gfns, int count)
  */
 if ( j < (PAGE_SIZE / sizeof(*map[i])) )
 {
-p2m_set_entry(p2m, _gfn(gfns[i]), mfns[i], PAGE_ORDER_4K,
+p2m_set_entry(p2m, gfns[i], mfns[i], PAGE_ORDER_4K,
   types[i], p2m->default_access);
 }
 else
@@ -952,7 +951,7 @@ p2m_pod_zero_check(struct p2m_domain *p2m, unsigned long 
*gfns, int count)
 int d:16,order:16;
 } t;
 
-t.gfn = gfns[i];
+t.gfn = gfn_x(gfns[i]);
 t.mfn = mfn_x(mfns[i]);
 t.d = d->domain_id;
 t.order = 0;
@@ -973,7 +972,7 @@ p2m_pod_zero_check(struct p2m_domain *p2m, unsigned long 
*gfns, int count)
 static void
 p2m_pod_emergency_sweep(struct p2m_domain *p2m)
 {
-unsigned long gfns[POD_SWEEP_STRIDE];
+gfn_t gfns[POD_SWEEP_STRIDE];
 unsigned long i, j = 0, start, limit;
 p2m_type_t t;
 
@@ -997,7 +996,7 @@ p2m_pod_emergency_sweep(struct p2m_domain *p2m)
 (void)p2m->get_entry(p2m, _gfn(i), , , 0, NULL, NULL);
 if ( p2m_is_ram(t) )
 {
-gfns[j] = i;
+gfns[j] = _gfn(i);
 j++;
 

[Xen-devel] [PATCH 00/15] xen/x86: Clean-up the PoD code

[Julien Grall]

Hi all

I have been attempting to use the PoD code on ARM (it will be sent in a
separate series) and spent sometimes to clean-up and switch to typesafe gfn
the current code.

The PoD code has been tested on ARM (the version is slightly different, mostly
renaming) and the x86 part as only been built test it.

Cheers,

Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Jan Beulich 
Cc: Jun Nakajima 
Cc: Kevin Tian 
Cc: Razvan Cojocaru 
Cc: Tamas K Lengyel 

Julien Grall (15):
  xen/x86: p2m-pod: Clean-up includes
  xen/x86: p2m-pod: Remove trailing whitespaces
  xen/x86: p2m-pod: Fix coding style for comments
  xen/x86: p2m-pod: Fix coding style
  xen/x86: p2m-pod: Avoid redundant assignments in
p2m_pod_demand_populate
  xen/x86: p2m-pod: Clean-up use of typesafe MFN
  xen/x86: p2m-pod: Use typesafe gfn in p2m_pod_decrease_reservation
  xen/x86: p2m: Use typesafe gfn for the P2M callbacks get_entry and
set_entry
  xen/x86: p2m: Use typesafe GFN in p2m_set_entry
  xen/x86: p2m-pod: Use typesafe GFN in pod_eager_record
  xen/x86: p2m-pod: Clean-up p2m_pod_zero_check
  xen/x86: p2m-pod: Use typesafe gfn in p2m_pod_zero_check
  xen/x86: p2m-pod: Use typesafe gfn in p2m_pod_demand_populate
  xen/x86: p2m-pod: Use typesafe gfn for the fields reclaim_single and
max_guest
  xen/x86: p2m-pod: Rework prototype of p2m_pod_demand_populate

 xen/arch/arm/p2m.c   |   3 +-
 xen/arch/x86/hvm/hvm.c   |   2 +-
 xen/arch/x86/mm/hap/nested_hap.c |   2 +-
 xen/arch/x86/mm/mem_access.c |  19 +-
 xen/arch/x86/mm/mem_sharing.c|   7 +-
 xen/arch/x86/mm/p2m-ept.c|  11 +-
 xen/arch/x86/mm/p2m-pod.c| 435 +--
 xen/arch/x86/mm/p2m-pt.c |  12 +-
 xen/arch/x86/mm/p2m.c| 139 +++--
 xen/common/memory.c  |   3 +-
 xen/include/asm-arm/p2m.h|  13 --
 xen/include/asm-x86/p2m.h|  23 +--
 xen/include/xen/p2m-common.h |  13 ++
 13 files changed, 370 insertions(+), 312 deletions(-)

-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 05/15] xen/x86: p2m-pod: Avoid redundant assignments in p2m_pod_demand_populate

[Julien Grall]

gfn_aligned is assigned 3 times with the exact same formula. All the
variables used are not modified, so consolidate in a single assignment
at the beginning of the function.

Signed-off-by: Julien Grall 

---

Cc: George Dunlap 
Cc: Jan Beulich 
Cc: Andrew Cooper 
---
 xen/arch/x86/mm/p2m-pod.c | 6 +-
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/xen/arch/x86/mm/p2m-pod.c b/xen/arch/x86/mm/p2m-pod.c
index f04d6e03e2..bcc87aee03 100644
--- a/xen/arch/x86/mm/p2m-pod.c
+++ b/xen/arch/x86/mm/p2m-pod.c
@@ -1079,7 +1079,7 @@ p2m_pod_demand_populate(struct p2m_domain *p2m, unsigned 
long gfn,
 {
 struct domain *d = p2m->domain;
 struct page_info *p = NULL; /* Compiler warnings */
-unsigned long gfn_aligned;
+unsigned long gfn_aligned = (gfn >> order) << order;
 mfn_t mfn;
 unsigned long i;
 
@@ -1102,7 +1102,6 @@ p2m_pod_demand_populate(struct p2m_domain *p2m, unsigned 
long gfn,
 if ( order == PAGE_ORDER_1G )
 {
 pod_unlock(p2m);
-gfn_aligned = (gfn >> order) << order;
 /*
  * Note that we are supposed to call p2m_set_entry() 512 times to
  * split 1GB into 512 2MB pages here. But We only do once here because
@@ -1147,8 +1146,6 @@ p2m_pod_demand_populate(struct p2m_domain *p2m, unsigned 
long gfn,
 
 BUG_ON((mfn_x(mfn) & ((1UL << order) - 1)) != 0);
 
-gfn_aligned = (gfn >> order) << order;
-
 p2m_set_entry(p2m, gfn_aligned, mfn, order, p2m_ram_rw,
   p2m->default_access);
 
@@ -1200,7 +1197,6 @@ remap_and_retry:
  * NOTE: In a p2m fine-grained lock scenario this might
  * need promoting the gfn lock from gfn->2M superpage.
  */
-gfn_aligned = (gfn >> order) << order;
 for ( i = 0; i < (1UL << order); i++ )
 p2m_set_entry(p2m, gfn_aligned + i, INVALID_MFN, PAGE_ORDER_4K,
   p2m_populate_on_demand, p2m->default_access);
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 02/15] xen/x86: p2m-pod: Remove trailing whitespaces

[Julien Grall]

Signed-off-by: Julien Grall 

---

Cc: George Dunlap 
Cc: Jan Beulich 
Cc: Andrew Cooper 
---
 xen/arch/x86/mm/p2m-pod.c | 46 +++---
 1 file changed, 23 insertions(+), 23 deletions(-)

diff --git a/xen/arch/x86/mm/p2m-pod.c b/xen/arch/x86/mm/p2m-pod.c
index fec87e5224..1f07441259 100644
--- a/xen/arch/x86/mm/p2m-pod.c
+++ b/xen/arch/x86/mm/p2m-pod.c
@@ -1,7 +1,7 @@
 /**
  * arch/x86/mm/p2m-pod.c
  *
- * Populate-on-demand p2m entries. 
+ * Populate-on-demand p2m entries.
  *
  * Copyright (c) 2009-2011 Citrix Systems, Inc.
  *
@@ -76,7 +76,7 @@ p2m_pod_cache_add(struct p2m_domain *p2m,
__func__, mfn_x(mfn), order, ((1UL << order) - 1));
 return -1;
 }
-
+
 for(i=0; i < 1 << order ; i++) {
 struct domain * od;
 
@@ -223,8 +223,8 @@ p2m_pod_set_cache_target(struct p2m_domain *p2m, unsigned 
long pod_target, int p
 /* If we can't allocate a superpage, try singleton pages */
 order = PAGE_ORDER_4K;
 goto retry;
-}   
-
+}
+
 printk("%s: Unable to allocate page for PoD cache (target=%lu 
cache=%ld)\n",
__func__, pod_target, p2m->pod.count);
 ret = -ENOMEM;
@@ -272,7 +272,7 @@ p2m_pod_set_cache_target(struct p2m_domain *p2m, unsigned 
long pod_target, int p
 
 if ( test_and_clear_bit(_PGT_pinned, &(page+i)->u.inuse.type_info) 
)
 put_page_and_type(page+i);
-
+
 if ( test_and_clear_bit(_PGC_allocated, &(page+i)->count_info) )
 put_page(page+i);
 
@@ -296,7 +296,7 @@ out:
  * definitions:
  * + M: static_max
  * + B: number of pages the balloon driver has ballooned down to.
- * + P: Number of populated pages. 
+ * + P: Number of populated pages.
  * + T: Old target
  * + T': New target
  *
@@ -311,10 +311,10 @@ out:
  *   the remainder of the ram to the guest OS.
  *  T count_info & PGC_allocated) != 0 ) 
- && ( (mfn_to_page(mfns[i])->count_info & 
(PGC_page_table|PGC_xen_heap)) == 0 ) 
+ && ( (mfn_to_page(mfns[i])->count_info & PGC_allocated) != 0 )
+ && ( (mfn_to_page(mfns[i])->count_info & 
(PGC_page_table|PGC_xen_heap)) == 0 )
  && ( (mfn_to_page(mfns[i])->count_info & PGC_count_mask) <= 
max_ref ) )
 map[i] = map_domain_page(mfns[i]);
 else
@@ -915,7 +915,7 @@ p2m_pod_zero_check(struct p2m_domain *p2m, unsigned long 
*gfns, int count)
 t.mfn = mfn_x(mfns[i]);
 t.d = d->domain_id;
 t.order = 0;
-
+
 __trace_var(TRC_MEM_POD_ZERO_RECLAIM, 0, sizeof(t), );
 }
 
@@ -924,7 +924,7 @@ p2m_pod_zero_check(struct p2m_domain *p2m, unsigned long 
*gfns, int count)
 p2m->pod.entry_count++;
 }
 }
-
+
 }
 
 #define POD_SWEEP_LIMIT 1024
@@ -1046,12 +1046,12 @@ p2m_pod_demand_populate(struct p2m_domain *p2m, 
unsigned long gfn,
 pod_lock(p2m);
 
 /* This check is done with the pod lock held.  This will make sure that
- * even if d->is_dying changes under our feet, p2m_pod_empty_cache() 
+ * even if d->is_dying changes under our feet, p2m_pod_empty_cache()
  * won't start until we're done. */
 if ( unlikely(d->is_dying) )
 goto out_fail;
 
-
+
 /* Because PoD does not have cache list for 1GB pages, it has to remap
  * 1GB region to 2MB chunks for a retry. */
 if ( order == PAGE_ORDER_1G )
@@ -1107,7 +1107,7 @@ p2m_pod_demand_populate(struct p2m_domain *p2m, unsigned 
long gfn,
 set_gpfn_from_mfn(mfn_x(mfn) + i, gfn_aligned + i);
 paging_mark_dirty(d, mfn_add(mfn, i));
 }
-
+
 p2m->pod.entry_count -= (1 << order);
 BUG_ON(p2m->pod.entry_count < 0);
 
@@ -1124,7 +1124,7 @@ p2m_pod_demand_populate(struct p2m_domain *p2m, unsigned 
long gfn,
 t.mfn = mfn_x(mfn);
 t.d = d->domain_id;
 t.order = order;
-
+
 __trace_var(TRC_MEM_POD_POPULATE, 0, sizeof(t), );
 }
 
@@ -1161,7 +1161,7 @@ remap_and_retry:
 
 t.gfn = gfn;
 t.d = d->domain_id;
-
+
 __trace_var(TRC_MEM_POD_SUPERPAGE_SPLINTER, 0, sizeof(t), );
 }
 
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] Windows "heinsenbug" (WAS: Re: Notes Design Session: Making Releases Lessons Learned: Improving Our Release Process and Tooling)

[Julien Grall]

Hi all,

On 07/27/2017 05:56 PM, Ian Jackson wrote:

Jan Beulich writes ("Re: Notes Design Session: Making Releases Lessons Learned: 
Improving Our Release Process and Tooling"):

[Lars:]

Ian: These are most likely software problems, most likely in Xen.
ISSUE: nobody wants to debug Windows Heisenbugs


Hmm, I think we believe to have understood what the issue is, it's just that
the change needed is on the tool stack side (gracefully deal with a domain
rebooting while being migrated).


No, the problem is that we don't understand why the domain is
rebooting.  It should not be.


This bug still seem to be around and likely going to impact once again 
the release testing.


As discussed during the summit, I am really thinking to make this 
"heinsenbug" a blocker until somehow finally spend sometimes 
investigating the problem and suggest actions.


Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC XEN PATCH v3 12/39] tools/xen-ndctl: add NVDIMM management util 'xen-ndctl'

[Dan Williams]

On Mon, Sep 11, 2017 at 2:24 PM, Konrad Rzeszutek Wilk
 wrote:
> On Mon, Sep 11, 2017 at 09:35:08AM -0700, Dan Williams wrote:
>> On Sun, Sep 10, 2017 at 10:39 PM, Haozhong Zhang
>>  wrote:
>> > On 09/10/17 22:10 -0700, Dan Williams wrote:
>> >> On Sun, Sep 10, 2017 at 9:37 PM, Haozhong Zhang
>> >>  wrote:
>> >> > The kernel NVDIMM driver and the traditional NVDIMM management
>> >> > utilities in Dom0 does not work now. 'xen-ndctl' is added as an
>> >> > alternatively, which manages NVDIMM via Xen hypercalls.
>> >> >
>> >> > Signed-off-by: Haozhong Zhang 
>> >> > ---
>> >> > Cc: Ian Jackson 
>> >> > Cc: Wei Liu 
>> >> > ---
>> >> >  .gitignore |   1 +
>> >> >  tools/misc/Makefile|   4 ++
>> >> >  tools/misc/xen-ndctl.c | 172 
>> >> > +
>> >> >  3 files changed, 177 insertions(+)
>> >> >  create mode 100644 tools/misc/xen-ndctl.c
>> >>
>> >> What about my offer to move this functionality into the upstream ndctl
>> >> utility [1]? I think it is thoroughly confusing that you are reusing
>> >> the name 'ndctl' and avoiding integration with the upstream ndctl
>> >> utility.
>> >>
>> >> [1]: https://patchwork.kernel.org/patch/9632865/
>> >
>> > I'm not object to integrate it with ndctl.
>> >
>> > My only concern is that the integration will introduces two types of
>> > user interface. The upstream ndctl works with the kernel driver and
>> > provides easily used *names* (e.g., namespace0.0, region0, nmem0,
>> > etc.) for user input. However, this version patchset hides NFIT from
>> > Dom0 (to simplify the first implementation), so the kernel driver does
>> > not work in Dom0, neither does ndctl. Instead, xen-ndctl has to use
>> > *the physical address* for users to specify their interested NVDIMM
>> > region, which is different from upstream ndctl.
>>
>> Ok, I think this means that xen-ndctl should be renamed (xen-nvdimm?)
>> so that the distinction between the 2 tools is clear.
>
> I think it makes much more sense to integrate this in the upstream
> version of ndctl. As surely in the future the ndctl will need to work
> with other OSes too? Such as FreeBSD?

I'm receptive to carrying Xen-specific enabling and / or a FreeBSD
compat layer in ndctl.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 07/17] livepatch/arm/x86: Strip note_depends symbol from test-cases.

[Konrad Rzeszutek Wilk]

On Wed, Sep 13, 2017 at 02:51:41AM -0600, Jan Beulich wrote:
> >>> On 13.09.17 at 01:46,  wrote:
> > On Tue, Sep 12, 2017 at 08:48:33AM -0600, Jan Beulich wrote:
> >> >>> On 12.09.17 at 02:37,  wrote:
> >> > This surfaced due to "xen/livepatch/x86/arm32: Force
> >> > .livepatch.depends section to be uint32_t aligned." which switched
> >> > to a different way of including the build-id.
> >> > 
> >> > Each livepatch ends with a global:
> >> > 
> >> > 30:  1 OBJECT  GLOBAL HIDDEN 7 note_depends
> >> > 
> >> > which will cause collision when loading.
> >> > 
> >> > One attempted solution was to add in the Makefile stanza:
> >> >  @sed -i '/unsigned/static unsinged/' $@
> >> > 
> >> > But that resulted in the note_depends being omitted from the livepatch
> >> > (as it was static and not used) which meant we would not have an
> >> > .livepatch_depends section which we require.
> >> 
> >> Did you consider using objcopy's --localize-symbol instead?
> > 
> > Yes, so that note_depends is not globally visible. But that won't help
> > as hypervisor treats both local and global symbols as global when resolving
> > them.
> > 
> > That is each of the livepatch has the node_depends in it, and we can't
> > load xen_hello_world, followed by xen_replace_world test-case (so
> > stacking them on top of each other) - as both have the same local
> > symbol.
> 
> Oh, right. Then perhaps stripping the symbol is as good or as bad as
> deriving the symbol name from e.g. the patch name, or putting some
> randomized tag on it.

Yes, and I had in mind changing the name of it (to prefix it with the
livepatch name) using --redefine-sym.

But then figured it may be just easier to ditch the symbol altogether.

Let me try it out - I do wonder if that would remove the need
for stripping the debug symbols or if that would still trip the issue
I keep on having - which is that the debug section would reference the original
symbol.


> 
> > (This is fixed in "livepatch: Add local and global symbol resolution."
> > on which you said:
> > 
> > > All the 'GLOBAL' have to be unique per livepatch. But the
> > > 'LOCAL' can all be the same which means the semantic of 'static'
> > > on functions and data variables is the right one.
> > 
> > I think this is wrong: Afaict your change results in main image and
> > patch local symbols to now be treated differently. While this may
> > indeed help patches which are meant to replace others, it is going
> > to get in the way if a patch wants to reference a local symbol
> > already altered (or newly introduced) by a prior one.
> > 
> > (https://www.mail-archive.com/xen-devel@lists.xen.org/msg111710.html)
> 
> Right, this is a basically unresolvable ambiguity, I'm afraid. We'd
> need a 3rd class of symbols. It may be worth considering to (ab)use
> e.g. STV_INTERNAL for this purpose.

Oh. Let me look at that. Thank you!
> 
> Jan
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC XEN PATCH v3 12/39] tools/xen-ndctl: add NVDIMM management util 'xen-ndctl'

[Konrad Rzeszutek Wilk]

On Mon, Sep 11, 2017 at 09:35:08AM -0700, Dan Williams wrote:
> On Sun, Sep 10, 2017 at 10:39 PM, Haozhong Zhang
>  wrote:
> > On 09/10/17 22:10 -0700, Dan Williams wrote:
> >> On Sun, Sep 10, 2017 at 9:37 PM, Haozhong Zhang
> >>  wrote:
> >> > The kernel NVDIMM driver and the traditional NVDIMM management
> >> > utilities in Dom0 does not work now. 'xen-ndctl' is added as an
> >> > alternatively, which manages NVDIMM via Xen hypercalls.
> >> >
> >> > Signed-off-by: Haozhong Zhang 
> >> > ---
> >> > Cc: Ian Jackson 
> >> > Cc: Wei Liu 
> >> > ---
> >> >  .gitignore |   1 +
> >> >  tools/misc/Makefile|   4 ++
> >> >  tools/misc/xen-ndctl.c | 172 
> >> > +
> >> >  3 files changed, 177 insertions(+)
> >> >  create mode 100644 tools/misc/xen-ndctl.c
> >>
> >> What about my offer to move this functionality into the upstream ndctl
> >> utility [1]? I think it is thoroughly confusing that you are reusing
> >> the name 'ndctl' and avoiding integration with the upstream ndctl
> >> utility.
> >>
> >> [1]: https://patchwork.kernel.org/patch/9632865/
> >
> > I'm not object to integrate it with ndctl.
> >
> > My only concern is that the integration will introduces two types of
> > user interface. The upstream ndctl works with the kernel driver and
> > provides easily used *names* (e.g., namespace0.0, region0, nmem0,
> > etc.) for user input. However, this version patchset hides NFIT from
> > Dom0 (to simplify the first implementation), so the kernel driver does
> > not work in Dom0, neither does ndctl. Instead, xen-ndctl has to use
> > *the physical address* for users to specify their interested NVDIMM
> > region, which is different from upstream ndctl.
> 
> Ok, I think this means that xen-ndctl should be renamed (xen-nvdimm?)
> so that the distinction between the 2 tools is clear.

I think it makes much more sense to integrate this in the upstream
version of ndctl. As surely in the future the ndctl will need to work
with other OSes too? Such as FreeBSD?

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] CONFIG_SCRUB_DEBUG=y + arm64 + livepatch = Xen BUG at page_alloc.c:738

[Konrad Rzeszutek Wilk]

On Tue, Sep 12, 2017 at 09:19:23PM -0400, Boris Ostrovsky wrote:
> 
> 
> On 09/12/2017 08:01 PM, Konrad Rzeszutek Wilk wrote:
> > On Mon, Sep 11, 2017 at 08:45:02PM -0400, Boris Ostrovsky wrote:
> > > 
> > > 
> > > On 09/11/2017 07:55 PM, Konrad Rzeszutek Wilk wrote:
> > > > Hey,
> > > > 
> > > > I've only been able to reproduce this on ARM64 (trying right now ARM32
> > > > as well), and not on x86.
> > > > 
> > > > If I compile Xen without CONFIG_SCRUB_DEBUG it works great. But if
> > > > enable it and try to load a livepatch it blows up in page_alloc.c:738
> > > > 
> > > > This is with origin/staging (d0291f3391)
> > > 
> > > Can you still reproduce this if you revert 307c3be?
> > 
> > Sadly yes - it still crashes. I didn't capture the serial output.
> > 
> > I honestly think the issue is that on ARM64 the "sleep" loop does not
> > wake up as often as on x86 (CC-ing Dariof who I believe observed this
> > with Credit2 and the wakeup.. something) - maybe he remembers the
> > details. Anyhow my theory is that the pages are not scrubbed at all
> > when they go in the idle loop as once it goes to sleep - it stays there.
> 
> 
> There is no (well, should not be) any timing dependencies in how/whether
> pages are scrubbed. If a page doesn't get scrubbed because someone didn't
> wake up then it should be scrubbed in alloc_heap_pages(). So in this case
> the page is thought to be clean (_PGC_need_scrub is not set), but it is not.
> 
> Have you tried running a guest (or two), rebooting in a loop?

No. I just cold-booted it and tried to livepatch.
> 
> Another thing to try is to set need_scrub to true in free_heap_pages().

Magic!

diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index dbad1e1ca0..9303eb4517 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -1308,6 +1308,7 @@ static void free_heap_pages(
 ASSERT(node >= 0);
 
 spin_lock(_lock);
+need_scrub = true;
 
 for ( i = 0; i < (1 << order); i++ )
 {

Fixes it ! :-)


> 
> -boris
> 
> 
> > 
> > Ah, see commit 05c52278a7c92bc753d9fe32017e4961012b9f23
> > 
> > Maybe this is related?
> > > 
> > > 
> > > -boris

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v6 01/12] xen: correct gnttab_get_status_frames()

[Paul Durrant]

> -Original Message-
> From: Juergen Gross [mailto:jgr...@suse.com]
> Sent: 13 September 2017 09:58
> To: Paul Durrant ; xen-devel@lists.xen.org
> Cc: sstabell...@kernel.org; Wei Liu ; George Dunlap
> ; Andrew Cooper
> ; Ian Jackson ; Tim
> (Xen.org) ; julien.gr...@arm.com; jbeul...@suse.com;
> dgde...@tycho.nsa.gov
> Subject: Re: [Xen-devel] [PATCH v6 01/12] xen: correct
> gnttab_get_status_frames()
> 
> On 13/09/17 18:01, Paul Durrant wrote:
> >> -Original Message-
> >> From: Xen-devel [mailto:xen-devel-boun...@lists.xen.org] On Behalf Of
> >> Juergen Gross
> >> Sent: 13 September 2017 08:47
> >> To: xen-devel@lists.xen.org
> >> Cc: Juergen Gross ; sstabell...@kernel.org; Wei Liu
> >> ; George Dunlap ;
> >> Andrew Cooper ; Ian Jackson
> >> ; Tim (Xen.org) ;
> >> julien.gr...@arm.com; jbeul...@suse.com; dgde...@tycho.nsa.gov
> >> Subject: [Xen-devel] [PATCH v6 01/12] xen: correct
> >> gnttab_get_status_frames()
> >>
> >> In gnttab_get_status_frames() all accesses to nr_status_frames should
> >> be done with the grant table lock held.
> >
> > Is this true? The value can only increase so what does the increase lock
> scope actually protect against?
> 
> The comment above nr_status_frames() says so. Either the comment or the
> code is wrong.

Ok. I suspect that was cut'n'paste from nr_grant_fames() but I see no 
particular harm in the increased scope since it's a read lock.

Reviewed-by: Paul Durrant  

> 
> 
> Juergen
> 
> >
> >   Paul
> >
> >>
> >> While at it correct coding style: labels should be indented by one
> >> space.
> >>
> >> Signed-off-by: Juergen Gross 
> >> ---
> >>  xen/common/grant_table.c | 15 ---
> >>  1 file changed, 8 insertions(+), 7 deletions(-)
> >>
> >> diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
> >> index c3895e6201..00ff075bd9 100644
> >> --- a/xen/common/grant_table.c
> >> +++ b/xen/common/grant_table.c
> >> @@ -2866,19 +2866,19 @@
> >>
> gnttab_get_status_frames(XEN_GUEST_HANDLE_PARAM(gnttab_get_statu
> >> s_frames_t) uop,
> >>
> >>  gt = d->grant_table;
> >>
> >> +op.status = GNTST_okay;
> >> +
> >> +grant_read_lock(gt);
> >> +
> >>  if ( unlikely(op.nr_frames > nr_status_frames(gt)) )
> >>  {
> >>  gdprintk(XENLOG_INFO, "Guest requested addresses for %d grant
> >> status "
> >>   "frames, but only %d are available.\n",
> >>   op.nr_frames, nr_status_frames(gt));
> >>  op.status = GNTST_general_error;
> >> -goto out2;
> >> +goto unlock;
> >>  }
> >>
> >> -op.status = GNTST_okay;
> >> -
> >> -grant_read_lock(gt);
> >> -
> >>  for ( i = 0; i < op.nr_frames; i++ )
> >>  {
> >>  gmfn = gnttab_status_gmfn(d, gt, i);
> >> @@ -2886,10 +2886,11 @@
> >>
> gnttab_get_status_frames(XEN_GUEST_HANDLE_PARAM(gnttab_get_statu
> >> s_frames_t) uop,
> >>  op.status = GNTST_bad_virt_addr;
> >>  }
> >>
> >> + unlock:
> >>  grant_read_unlock(gt);
> >> -out2:
> >> + out2:
> >>  rcu_unlock_domain(d);
> >> -out1:
> >> + out1:
> >>  if ( unlikely(__copy_field_to_guest(uop, , status)) )
> >>  return -EFAULT;
> >>
> >> --
> >> 2.12.3
> >>
> >>
> >> ___
> >> Xen-devel mailing list
> >> Xen-devel@lists.xen.org
> >> https://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v6 04/12] xen: add new domctl hypercall to set grant table resource limits

[Daniel De Graaf]

On 09/13/2017 11:46 AM, Juergen Gross wrote:

Add a domctl hypercall to set the domain's resource limits regarding
grant tables. It is accepted only as long as neither
gnttab_setup_table() has been called for the domain, nor the domain
has started to run.

Signed-off-by: Juergen Gross 


Acked-by: Daniel De Graaf 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [distros-debian-squeeze test] 72101: regressions - trouble: blocked/broken/fail/pass

[Platform Team regression test user]

flight 72101 distros-debian-squeeze real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/72101/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-pvops 6 kernel-build  fail REGR. vs. 72066

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-amd64-squeeze-netboot-pygrub  1 build-check(1)blocked n/a
 test-amd64-amd64-i386-squeeze-netboot-pygrub  1 build-check(1) blocked n/a
 build-arm64-pvops 2 hosts-allocate   broken like 72066
 build-arm64   2 hosts-allocate   broken like 72066
 build-arm64   3 capture-logs broken like 72066
 build-arm64-pvops 3 capture-logs broken like 72066
 test-amd64-i386-amd64-squeeze-netboot-pygrub 10 debian-di-install fail like 
72066
 test-amd64-i386-i386-squeeze-netboot-pygrub 10 debian-di-install fail like 
72066

baseline version:
 flight   72066

jobs:
 build-amd64  pass
 build-arm64  broken  
 build-armhf  pass
 build-i386   pass
 build-amd64-pvopsfail
 build-arm64-pvopsbroken  
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-amd64-squeeze-netboot-pygrubblocked 
 test-amd64-i386-amd64-squeeze-netboot-pygrub fail
 test-amd64-amd64-i386-squeeze-netboot-pygrub blocked 
 test-amd64-i386-i386-squeeze-netboot-pygrub  fail



sg-report-flight on osstest.xs.citrite.net
logs: /home/osstest/logs
images: /home/osstest/images

Logs, config files, etc. are available at
http://osstest.xs.citrite.net/~osstest/testlogs/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Push not applicable.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 04/17] x86emul: support F16C insns

[George Dunlap]

On Wed, Jun 21, 2017 at 1:01 PM, Jan Beulich  wrote:
> Note that this avoids emulating the behavior of VCVTPS2PH found on at
> least some Intel CPUs, which update MXCSR even when the memory write
> faults.
>
> Signed-off-by: Jan Beulich 
>
> --- a/tools/tests/x86_emulator/test_x86_emulator.c
> +++ b/tools/tests/x86_emulator/test_x86_emulator.c
> @@ -3028,6 +3028,47 @@ int main(int argc, char **argv)
>  printf("skipped\n");
>  #endif
>
> +printf("%-40s", "Testing vcvtph2ps (%ecx),%ymm1...");
> +if ( stack_exec && cpu_has_f16c )
> +{
> +decl_insn(vcvtph2ps);
> +decl_insn(vcvtps2ph);
> +
> +asm volatile ( "vxorps %%xmm1, %%xmm1, %%xmm1\n"
> +   put_insn(vcvtph2ps, "vcvtph2ps (%0), %%ymm1")
> +   :: "c" (NULL) );
> +
> +set_insn(vcvtph2ps);
> +res[1] = 0x40003c00; /* (1.0, 2.0) */
> +res[2] = 0x44004200; /* (3.0, 4.0) */
> +res[3] = 0x3400b800; /* (-.5, .25) */
> +res[4] = 0xbc00; /* (0.0, -1.) */
> +memset(res + 5, 0xff, 16);
> +regs.ecx = (unsigned long)(res + 1);
> +rc = x86_emulate(, );
> +asm volatile ( "vmovups %%ymm1, %0" : "=m" (res[16]) );
> +if ( rc != X86EMUL_OKAY || !check_eip(vcvtph2ps) )
> +goto fail;
> +printf("okay\n");
> +
> +printf("%-40s", "Testing vcvtps2ph $0,%ymm1,(%edx)...");
> +asm volatile ( "vmovups %0, %%ymm1\n"
> +   put_insn(vcvtps2ph, "vcvtps2ph $0, %%ymm1, (%1)")
> +   :: "m" (res[16]), "d" (NULL) );
> +
> +set_insn(vcvtps2ph);
> +memset(res + 7, 0, 32);
> +regs.edx = (unsigned long)(res + 7);
> +rc = x86_emulate(, );
> +if ( rc != X86EMUL_OKAY || !check_eip(vcvtps2ph) ||
> + memcmp(res + 1, res + 7, 16) ||
> + res[11] || res[12] || res[13] || res[14] )
> +goto fail;
> +printf("okay\n");
> +}
> +else
> +printf("skipped\n");
> +
>  #undef decl_insn
>  #undef put_insn
>  #undef set_insn
> --- a/tools/tests/x86_emulator/x86_emulate.h
> +++ b/tools/tests/x86_emulator/x86_emulate.h
> @@ -127,6 +127,14 @@ static inline uint64_t xgetbv(uint32_t x
>  (res.c & (1U << 28)) != 0; \
>  })
>
> +#define cpu_has_f16c ({ \
> +struct cpuid_leaf res; \
> +emul_test_cpuid(1, 0, , NULL); \
> +if ( !(res.c & (1U << 27)) || ((xgetbv(0) & 6) != 6) ) \
> +res.c = 0; \
> +(res.c & (1U << 29)) != 0; \
> +})
> +
>  #define cpu_has_avx2 ({ \
>  struct cpuid_leaf res; \
>  emul_test_cpuid(1, 0, , NULL); \
> --- a/xen/arch/x86/x86_emulate/x86_emulate.c
> +++ b/xen/arch/x86/x86_emulate/x86_emulate.c
> @@ -369,6 +369,7 @@ static const struct {
>  [0x00 ... 0x0b] = { .simd_size = simd_packed_int },
>  [0x0c ... 0x0f] = { .simd_size = simd_packed_fp },
>  [0x10] = { .simd_size = simd_packed_int },
> +[0x13] = { .simd_size = simd_other, .two_op = 1 },
>  [0x14 ... 0x15] = { .simd_size = simd_packed_fp },
>  [0x17] = { .simd_size = simd_packed_int, .two_op = 1 },
>  [0x18 ... 0x19] = { .simd_size = simd_scalar_fp, .two_op = 1 },
> @@ -411,6 +412,7 @@ static const struct {
>  [0x14 ... 0x17] = { .simd_size = simd_none, .to_mem = 1, .two_op = 1 },
>  [0x18] = { .simd_size = simd_128 },
>  [0x19] = { .simd_size = simd_128, .to_mem = 1, .two_op = 1 },
> +[0x1d] = { .simd_size = simd_other, .to_mem = 1, .two_op = 1 },
>  [0x20] = { .simd_size = simd_none },
>  [0x21] = { .simd_size = simd_other },
>  [0x22] = { .simd_size = simd_none },
> @@ -1601,6 +1603,7 @@ static bool vcpu_has(
>  #define vcpu_has_popcnt()  vcpu_has( 1, ECX, 23, ctxt, ops)
>  #define vcpu_has_aesni()   vcpu_has( 1, ECX, 25, ctxt, ops)
>  #define vcpu_has_avx() vcpu_has( 1, ECX, 28, ctxt, ops)
> +#define vcpu_has_f16c()vcpu_has( 1, ECX, 29, ctxt, ops)
>  #define vcpu_has_rdrand()  vcpu_has( 1, ECX, 30, ctxt, ops)
>  #define vcpu_has_mmxext() (vcpu_has(0x8001, EDX, 22, ctxt, ops) || \
> vcpu_has_sse())
> @@ -7216,6 +7219,12 @@ x86_emulate(
>  host_and_vcpu_must_have(sse4_1);
>  goto simd_0f38_common;
>
> +case X86EMUL_OPC_VEX_66(0x0f38, 0x13): /* vcvtph2ps xmm/mem,{x,y}mm */
> +generate_exception_if(vex.w, EXC_UD);
> +host_and_vcpu_must_have(f16c);
> +op_bytes = 8 << vex.l;
> +goto simd_0f_ymm;
> +
>  case X86EMUL_OPC_VEX_66(0x0f38, 0x20): /* vpmovsxbw xmm/mem,{x,y}mm */
>  case X86EMUL_OPC_VEX_66(0x0f38, 0x21): /* vpmovsxbd xmm/mem,{x,y}mm */
>  case X86EMUL_OPC_VEX_66(0x0f38, 0x22): /* vpmovsxbq xmm/mem,{x,y}mm */
> @@ -7607,6 +7616,50 @@ x86_emulate(
>  opc = init_prefixes(stub);
>  goto pextr;
>
> +case X86EMUL_OPC_VEX_66(0x0f3a, 0x1d): /* vcvtps2ph 
> $imm8,{x,y}mm,xmm/mem */

On the whole 

Re: [Xen-devel] [PATCH v6 01/12] xen: correct gnttab_get_status_frames()

[Juergen Gross]

On 13/09/17 18:01, Paul Durrant wrote:
>> -Original Message-
>> From: Xen-devel [mailto:xen-devel-boun...@lists.xen.org] On Behalf Of
>> Juergen Gross
>> Sent: 13 September 2017 08:47
>> To: xen-devel@lists.xen.org
>> Cc: Juergen Gross ; sstabell...@kernel.org; Wei Liu
>> ; George Dunlap ;
>> Andrew Cooper ; Ian Jackson
>> ; Tim (Xen.org) ;
>> julien.gr...@arm.com; jbeul...@suse.com; dgde...@tycho.nsa.gov
>> Subject: [Xen-devel] [PATCH v6 01/12] xen: correct
>> gnttab_get_status_frames()
>>
>> In gnttab_get_status_frames() all accesses to nr_status_frames should
>> be done with the grant table lock held.
> 
> Is this true? The value can only increase so what does the increase lock 
> scope actually protect against?

The comment above nr_status_frames() says so. Either the comment or the
code is wrong.


Juergen

> 
>   Paul
> 
>>
>> While at it correct coding style: labels should be indented by one
>> space.
>>
>> Signed-off-by: Juergen Gross 
>> ---
>>  xen/common/grant_table.c | 15 ---
>>  1 file changed, 8 insertions(+), 7 deletions(-)
>>
>> diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
>> index c3895e6201..00ff075bd9 100644
>> --- a/xen/common/grant_table.c
>> +++ b/xen/common/grant_table.c
>> @@ -2866,19 +2866,19 @@
>> gnttab_get_status_frames(XEN_GUEST_HANDLE_PARAM(gnttab_get_statu
>> s_frames_t) uop,
>>
>>  gt = d->grant_table;
>>
>> +op.status = GNTST_okay;
>> +
>> +grant_read_lock(gt);
>> +
>>  if ( unlikely(op.nr_frames > nr_status_frames(gt)) )
>>  {
>>  gdprintk(XENLOG_INFO, "Guest requested addresses for %d grant
>> status "
>>   "frames, but only %d are available.\n",
>>   op.nr_frames, nr_status_frames(gt));
>>  op.status = GNTST_general_error;
>> -goto out2;
>> +goto unlock;
>>  }
>>
>> -op.status = GNTST_okay;
>> -
>> -grant_read_lock(gt);
>> -
>>  for ( i = 0; i < op.nr_frames; i++ )
>>  {
>>  gmfn = gnttab_status_gmfn(d, gt, i);
>> @@ -2886,10 +2886,11 @@
>> gnttab_get_status_frames(XEN_GUEST_HANDLE_PARAM(gnttab_get_statu
>> s_frames_t) uop,
>>  op.status = GNTST_bad_virt_addr;
>>  }
>>
>> + unlock:
>>  grant_read_unlock(gt);
>> -out2:
>> + out2:
>>  rcu_unlock_domain(d);
>> -out1:
>> + out1:
>>  if ( unlikely(__copy_field_to_guest(uop, , status)) )
>>  return -EFAULT;
>>
>> --
>> 2.12.3
>>
>>
>> ___
>> Xen-devel mailing list
>> Xen-devel@lists.xen.org
>> https://lists.xen.org/xen-devel


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC] Unicore Subproject Proposal

[Lars Kurth]

Simon,

It looks to me as if there is some feedback: so it may make some sense to 
incorporate some of it and send out a version 2. We may also want to CC some 
reps from other unikernel projects but Mirage OS. Or you could point them to 
this thread in a separate mail through respective channels or a hub like 
unikernel.org. Whatever you think may work best. It’s your call.

And then have a formal vote, a week after v2 of the proposal. Does this work?

Lars

On 11/09/2017, 05:08, "Simon Kuenzer"  wrote:

Hi Alexander,

thanks a lot for your review.

On 10.09.2017 22:48, Alexander Dubinin wrote:
> Hi Felipe, all,
> 
> Great that it's going to start :) Looking forward to join :)

I am looking forward to your contributions. ;)

> 
> Just my 2 cents:
> 
> 1. Is this academic project, or it have specific goals and areas of 
> application? Would be good to have some practical use-cases and well 
> formulated list of problems (we all feel these by guts, but...), it 
> aiming to solve. IMHO that will help to prioritize functionality and get 
> usable result faster :)

It is kind of both, however we aim a strong focus on real world 
problems: IoT, Mobile Edge Computing (MEC), Automotive, Virtual Network 
Functions (VNFs), and others.
We have played with many Unikernels (ClickOS, Mirage, Rump, OSv, and 
others) and tried to apply them in the several areas. While doing this, 
we noticed that each area benefits differently from the properties that 
Unikernels give - which is great (e.g., instant boot times for MEC, high 
performance for NFV, resource efficiency for IoT). However, building and 
maintaining new Unikernels (as we did with ClickOS, MiniCache, and 
Minipython) is currently painful.
Because of different focuses on properties and ported/implemented 
applications, most Unikernel today are bound to their own OS layers 
(e.g., ClickOS uses a different Mini-OS than Mirage). Each application 
requires a different subset of OS layers but also enables different 
optimizations of them.

In order to solve this, we came up with the Unicore proposal. But I 
agree with your suggestion at this point: It helps for the project start 
to focus on some initial areas. For now, I hope this is driven by the 
first contributors, and I have personally IoT in mind. Since the project 
goal is so ambitious, we should keep the long-term goal in mind from the 
beginning.

> 
> 2. Does any security subsystem planned? XEN have XSM/FLASK, but IMHO is 
> should be supplemented by some security layer in control/stub domains as 
> well. So far only known implementation is OpenXT, but it is very 
> specific. Probably some generalized security layer needed in Unicore to 
> supplement FLASK/XSM... Correct me please, if I misunderstanding :)

I agree that many projects (especially embedded, stubdomains, driver 
domains, NFV) have a vested interest in security and isolation. In my 
view, XSM/FLASK further restricts what a VM can do and sounds kind of 
orthogonal to the functionality of a VM (am I right?). The fact that 
Unikernels should only pick components that are actually required to do 
the job reduces the attack surface compared to general purpose OSes.
Do you see further value with FLASK/XSM which requires early 
implementation and design decisions for Unicore? As far as I can tell 
something like Flask is implemented mostly in the hypervisor and 
toolstack, not in the guests themselves, is this right?


Thanks,

Simon

> 
> Regards,
>Alexander
> 
> On Fri, Sep 8, 2017 at 3:31 PM, Felipe Huici  > wrote:
> 
> Hi Wei, Stefano,
> 
> Thank you so much for agreeing to be sponsors! I’ll update the 
document.
> 
> — Felipe
> 
> 
> Dr. Felipe Huici
> Chief Researcher, Networked Systems and Data
> Analytics Group
> NEC Laboratories Europe, Network Research Division
> Kurfuerstenanlage 36, D-69115 Heidelberg
> Tel. +49
> (0)6221 4342-241
> Fax: +49
> (0)6221 4342-155
> 
> e-mail:
> felipe.hu...@neclab.eu 
> 
> NEC Europe Limited Registered Office: NEC House, 1
> Victoria Road, London W3 6BL Registered in England 2832014
> 
> 
> 
> 
> On 9/8/17, 1:00 PM, "Lars Kurth"  > wrote:
> 
>  >@Wei, @Stefano,
>  >
>  >On 07/09/2017, 22:16, "Stefano 

[Xen-devel] [xen-unstable-smoke test] 113414: regressions - FAIL

[osstest service owner]

flight 113414 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/113414/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemuu-debianhvm-i386 15 guest-saverestore.2 fail REGR. vs. 
113384

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  20b1a022558b02d9fb23df5b98fb442037ac0572
baseline version:
 xen  082fc63f20e827eb0229d520b4ebf54140d9b21b

Last test of basis   113384  2017-09-12 23:14:17 Z0 days
Failing since113403  2017-09-13 09:03:32 Z0 days3 attempts
Testing same since   113410  2017-09-13 12:23:19 Z0 days2 attempts


People who touched revisions under test:
  Andrew Cooper 
  Haozhong Zhang 
  Jan Beulich 
  Oleksandr Grytsov 
  Petre Pircalabu 
  Wei Liu 

jobs:
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-amd64-amd64-xl-qemuu-debianhvm-i386 fail
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 332 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH] xen: Credit2: enable fully custom runqueue arrangement

[Dario Faggioli]

The patch introduces yet another runqueue arrangement option
for Credit2. In fact, it allows the user to specify, explicitly
and precisely, what pCPUs should belong to which runqueue.

Signed-off-by: Dario Faggioli 
Signed-off-by: Praveen Kumar 
---
Cc: George Dunlap 
Cc: Praveen Kumar 
---
This is derived --although *heavily* reworked-- from what Praveen
has submitted before, here:

 https://lists.xen.org/archives/html/xen-devel/2017-04/msg02402.html
 https://lists.xen.org/archives/html/xen-devel/2017-06/msg00201.html

During my review of that, I suggested to make the array dynamically
allocated, using nr_cpu_ids as its size. Turns out, that does not
make much sense, as at the time the parameters are parsed, nr_cpu_ids
is still equal to NR_CPUS.
---
 docs/misc/xen-command-line.markdown |   12 +++
 xen/common/sched_credit2.c  |  135 +++
 2 files changed, 146 insertions(+), 1 deletion(-)

diff --git a/docs/misc/xen-command-line.markdown 
b/docs/misc/xen-command-line.markdown
index 9797c8d..dbf5d4c 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -567,7 +567,7 @@ also slow in responding to load changes.
 The default value of `1 sec` is rather long.
 
 ### credit2\_runqueue
-> `= cpu | core | socket | node | all`
+> `= cpu | core | socket | node | all | `
 
 > Default: `socket`
 
@@ -585,6 +585,16 @@ Available alternatives, with their meaning, are:
 * `node`: one runqueue per each NUMA node of the host;
 * `all`: just one runqueue shared by all the logical pCPUs of
  the host
+* ``: one runqueue per each specified subset of logical
+  pCPUs of the host. Subsets are defined either as:
+  `[[0,1,][2,6][3,5][4,7]]`, or as:
+  `'0,1;2,6;3,5;4,7'`. That means
+   - pCPUs 0 and 1 belong to runqueue 0
+   - pCPUs 2 and 6 belong to runqueue 1
+   - pCPUs 3 and 5 belong to runqueue 2
+   - pCPUs 4 and 7 belong to runqueue 3
+  pCPUs that are present on the host, but are not
+  part of any subset, are assigned to runqueue 0.
 
 ### dbgp
 > `= ehci[  | @pci:. ]`
diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c
index 0f93ad5..10da084 100644
--- a/xen/common/sched_credit2.c
+++ b/xen/common/sched_credit2.c
@@ -12,6 +12,7 @@
 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -323,6 +324,17 @@ integer_param("credit2_balance_over", 
opt_overload_balance_tolerance);
  *   (logical) processors of the host belong. This will happen if
  *   the opt_runqueue parameter is set to 'all'.
  *
+ * - custom: meaning that there will be one runqueue per each specified
+ *   subset, as shown in the following examples:
+ *   - credit2_runqueue=[[0,1][3][4,5]]
+ *   - credit2_runqueue='0,1;3;4,5'
+ *   These (both) mean the following:
+ *   - CPU 0 and CPU 1 belong to runqueue 0
+ *   - CPU 3 belongs to runqueue 1
+ *   - CPU 4 and CPU 5 belong to runqueue 2
+ *   CPUs that are present on the host, but are not part of any
+ *   defined subset, will be assigned to runqueue 0.
+ *
  * Depending on the value of opt_runqueue, therefore, cpus that are part of
  * either the same physical core, the same physical socket, the same NUMA
  * node, or just all of them, will be put together to form runqueues.
@@ -332,6 +344,7 @@ integer_param("credit2_balance_over", 
opt_overload_balance_tolerance);
 #define OPT_RUNQUEUE_SOCKET 2
 #define OPT_RUNQUEUE_NODE   3
 #define OPT_RUNQUEUE_ALL4
+#define OPT_RUNQUEUE_CUSTOM 5
 static const char *const opt_runqueue_str[] = {
 [OPT_RUNQUEUE_CPU] = "cpu",
 [OPT_RUNQUEUE_CORE] = "core",
@@ -341,6 +354,115 @@ static const char *const opt_runqueue_str[] = {
 };
 static int __read_mostly opt_runqueue = OPT_RUNQUEUE_SOCKET;
 
+static unsigned int __read_mostly custom_cpu_runqueue[NR_CPUS];
+
+static int parse_custom_runqueue(const char *s)
+{
+unsigned int cpu, rqi = 0;
+bool in_subset = false, ret = true;
+cpumask_t cpus;
+
+/*
+ * If we are dealing with format 1 (i.e., [[0,1][3,5]]), first
+ * and last character must be '[' and ']', respectively.
+ *
+ * If we are dealing with format 2 (i.e., 0,1;3,5), first and
+ * last character must be numbers.
+ */
+if ( *s == '[' && *(s+strlen(s)-1) == ']' )
+s++;
+else if ( !(isdigit(*s) && isdigit(s[strlen(s)-1])) )
+return false;
+
+cpumask_clear();
+while ( !(*s == ']' && *(s+1) == '\0') && *s != '\0' )
+{
+/*
+ * We tolerate only the allowed characters (depending on the
+ * format). Also, we don't accept empty subsets.
+ */
+if ( *(s+strlen(s)-1) == ']' )
+{
+/* Format 1 */
+if ( !(*s == 

Re: [Xen-devel] [PATCH 2/4] xen: limit grant v2 interface to the v1 functionality

[Boris Ostrovsky]

On 09/13/2017 10:45 AM, Juergen Gross wrote:
> On 13/09/17 15:50, Boris Ostrovsky wrote:
>> On 09/13/2017 09:38 AM, Juergen Gross wrote:
>>> On 13/09/17 15:22, Boris Ostrovsky wrote:
 On 09/12/2017 02:18 PM, Juergen Gross wrote:
> On 12/09/17 18:21, Boris Ostrovsky wrote:
>> On 09/12/2017 12:09 PM, Juergen Gross wrote:
>>> On 12/09/17 18:05, Boris Ostrovsky wrote:
 On 09/12/2017 11:50 AM, Juergen Gross wrote:
> On 12/09/17 17:44, Boris Ostrovsky wrote:
>> On 09/08/2017 10:48 AM, Juergen Gross wrote:
>>> As there is currently no user for sub-page grants or transient 
>>> grants
>>> remove that functionality. This at once makes it possible to switch
>>> from grant v2 to grant v1 without restrictions, as there is no loss 
>>> of
>>> functionality other than the limited frame number width related to
>>> the switch.
>> But isn't that ABI violation? v2 is expected to support this (XSAs
>> notwithstanding)
> No, I don't think so.
>
> The hypervisor still supports it, but the domU (or dom0) isn't 
> required
> to make use of all the features IMHO. Or are you aware of any backend
> querying the grant version of a frontend and acting in another way if 
> v2
> is detected?
 I am not aware of any but that doesn't mean that they don't (or won't)
 exist.
>>> But isn't the frontend the one which is defining what is granted in
>>> which way? How should there be an ABI breakage when the frontend just
>>> isn't using sub-page or transitive grants?
>> People may provide both front and backend drivers and frontends, knowing
>> that v2 is available, could decide to use those features.
> No, without the functions to use them it will be impossible.
 I don't follow this. Which functions? The ones this patch is removing?
>>> Yes, just after having been added one patch earlier.
>>>
>>> Right now the Linux kernel doesn't support grant V2 at all. So there
>>> surely is no driver making use of V2 features right now.
>>>
>>> Ican merge patches 1 and 2 in case you want. I just thought a pure
>>> revert of the former V2 remove patch would be easier to review,
>>> taking into account that the former V2 support was working in
>>> production environments (and even back then there was no user of
>>> sub-page or transient grants).
>> No, I don't have problems with *how* you are doing this (revert fully
>> first and then remove).
>>
>> I am just not sure that removing these functions is the way to go
>> because we are ending up with partial implementation of v2. The fact
>> that noone is/has been using these features is IMO not particularly
>> relevant.
>>
>> If these two were optional features then it would have been reasonable
>> to drop them.
> Why does the kernel need to support all features of an interface?
>
> I'm quite sure there are lots of interfaces supported only partially in
> the kernel.

I don't think partially supported interface is a supported interface.
It's just something that has been working until now.

> Having support for functionality in the kernel not being used at all is
> just adding dead code with a high potential to bitrot. I can't even test
> this functionality right now, as there are no users of it. So I'd risk
> adding something which is broken from the beginning. 

OK. That I haven't considered that.

BTW, why are you not removing (*update_trans_entry) definition from
gnttab_ops? You are taking (*update_subpage_entry) out.


> So currently my V2
> support is regarding the exported kernel interfaces the same as V1, but
> without the limitation to 32 bit frame numbers.
>
> And looking at
>
> https://lists.xen.org/archives/html/xen-devel/2017-08/msg03194.html
>
> I believe my partial V2 support isn't the worst idea.

I never said that ;-)

-boris


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Minios-devel] [Xen-API] [RFC] Unicore Subproject Proposal

[Samuel Thibault]

Hello,

Anil Madhavapeddy, on mer. 13 sept. 2017 11:11:03 +0100, wrote:
> Maintaining a forked MiniOS has been a multi-year source of a maintenance 
> burden for MirageOS,

I'm just wondering why this happened?

The mini-os repository is open for development, it's just a matter of
agreeing on how to implement features :)

> One requirement from our side is that we need to strip down MiniOS to remove 
> even the C xenstore implementation, since we have pure OCaml gnt, xenstore 
> and device driver implementations.

That could already be a build option in MiniOS.  It seems it actually
already is, via $(CONFIG_XENBUS)=n.

If launching UniCore allows to get momentum to achieve that, I'm all for
it, I'm just wondering whether the problem is not actually about putting
a name on it, but just discussing here what is needed, and see how to
implement it all within the same repository. But again, I understand
that putting a name can trigger that discussion and be a "let's do it!"
message :)

Samuel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v6 01/12] xen: correct gnttab_get_status_frames()

[Paul Durrant]

> -Original Message-
> From: Xen-devel [mailto:xen-devel-boun...@lists.xen.org] On Behalf Of
> Juergen Gross
> Sent: 13 September 2017 08:47
> To: xen-devel@lists.xen.org
> Cc: Juergen Gross ; sstabell...@kernel.org; Wei Liu
> ; George Dunlap ;
> Andrew Cooper ; Ian Jackson
> ; Tim (Xen.org) ;
> julien.gr...@arm.com; jbeul...@suse.com; dgde...@tycho.nsa.gov
> Subject: [Xen-devel] [PATCH v6 01/12] xen: correct
> gnttab_get_status_frames()
> 
> In gnttab_get_status_frames() all accesses to nr_status_frames should
> be done with the grant table lock held.

Is this true? The value can only increase so what does the increase lock scope 
actually protect against?

  Paul

> 
> While at it correct coding style: labels should be indented by one
> space.
> 
> Signed-off-by: Juergen Gross 
> ---
>  xen/common/grant_table.c | 15 ---
>  1 file changed, 8 insertions(+), 7 deletions(-)
> 
> diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
> index c3895e6201..00ff075bd9 100644
> --- a/xen/common/grant_table.c
> +++ b/xen/common/grant_table.c
> @@ -2866,19 +2866,19 @@
> gnttab_get_status_frames(XEN_GUEST_HANDLE_PARAM(gnttab_get_statu
> s_frames_t) uop,
> 
>  gt = d->grant_table;
> 
> +op.status = GNTST_okay;
> +
> +grant_read_lock(gt);
> +
>  if ( unlikely(op.nr_frames > nr_status_frames(gt)) )
>  {
>  gdprintk(XENLOG_INFO, "Guest requested addresses for %d grant
> status "
>   "frames, but only %d are available.\n",
>   op.nr_frames, nr_status_frames(gt));
>  op.status = GNTST_general_error;
> -goto out2;
> +goto unlock;
>  }
> 
> -op.status = GNTST_okay;
> -
> -grant_read_lock(gt);
> -
>  for ( i = 0; i < op.nr_frames; i++ )
>  {
>  gmfn = gnttab_status_gmfn(d, gt, i);
> @@ -2886,10 +2886,11 @@
> gnttab_get_status_frames(XEN_GUEST_HANDLE_PARAM(gnttab_get_statu
> s_frames_t) uop,
>  op.status = GNTST_bad_virt_addr;
>  }
> 
> + unlock:
>  grant_read_unlock(gt);
> -out2:
> + out2:
>  rcu_unlock_domain(d);
> -out1:
> + out1:
>  if ( unlikely(__copy_field_to_guest(uop, , status)) )
>  return -EFAULT;
> 
> --
> 2.12.3
> 
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v5 04/11] x86/physdev: enable PHYSDEVOP_pci_mmcfg_reserved for PVH Dom0

[Roger Pau Monné]

On Tue, Sep 05, 2017 at 08:57:54AM -0600, Jan Beulich wrote:
> >>> On 14.08.17 at 16:28,  wrote:
> > --- a/xen/arch/x86/physdev.c
> > +++ b/xen/arch/x86/physdev.c
> > @@ -559,6 +559,15 @@ ret_t do_physdev_op(int cmd, 
> > XEN_GUEST_HANDLE_PARAM(void) arg)
> >  
> >  ret = pci_mmcfg_reserved(info.address, info.segment,
> >   info.start_bus, info.end_bus, info.flags);
> > +if ( ret || !is_hvm_domain(currd) )
> > +break;
> 
> Don't you also want to check has_vpci() here?

I don't think the also is needed here, just checking for has_vpci
should be fine (PV guests will not have the vpci flag set in any
case).

Thanks, Roger.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v6 02/12] xen: move XENMAPSPACE_grant_table code into grant_table.c

[Juergen Gross]

The x86 and arm versions of XENMAPSPACE_grant_table handling are nearly
identical. Move the code into a function in grant_table.c and add an
architecture dependant hook to handle the differences.

Switch to mfn_t in order to be more type safe.

Signed-off-by: Juergen Gross 
Reviewed-by: Paul Durrant 
Reviewed-by: Wei Liu 
---
V6:
- test rc of gnttab_map_frame() (Jan Beulich)
V3:
- update commit message

V2:
- rebased to staging
---
 xen/arch/arm/mm.c | 36 
 xen/arch/x86/mm.c | 43 +++
 xen/common/grant_table.c  | 38 ++
 xen/include/asm-arm/grant_table.h |  7 +++
 xen/include/asm-x86/grant_table.h |  5 +
 xen/include/xen/grant_table.h |  3 +++
 6 files changed, 69 insertions(+), 63 deletions(-)

diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c
index b39677eac9..3db0e3bdea 100644
--- a/xen/arch/arm/mm.c
+++ b/xen/arch/arm/mm.c
@@ -1229,39 +1229,11 @@ int xenmem_add_to_physmap_one(
 switch ( space )
 {
 case XENMAPSPACE_grant_table:
-grant_write_lock(d->grant_table);
-
-if ( d->grant_table->gt_version == 0 )
-d->grant_table->gt_version = 1;
-
-if ( d->grant_table->gt_version == 2 &&
-(idx & XENMAPIDX_grant_table_status) )
-{
-idx &= ~XENMAPIDX_grant_table_status;
-if ( idx < nr_status_frames(d->grant_table) )
-mfn = virt_to_mfn(d->grant_table->status[idx]);
-}
-else
-{
-if ( (idx >= nr_grant_frames(d->grant_table)) &&
- (idx < max_grant_frames) )
-gnttab_grow_table(d, idx + 1);
-
-if ( idx < nr_grant_frames(d->grant_table) )
-mfn = virt_to_mfn(d->grant_table->shared_raw[idx]);
-}
-
-if ( !mfn_eq(mfn, INVALID_MFN) )
-{
-d->arch.grant_table_gfn[idx] = gfn;
-
-t = p2m_ram_rw;
-}
-
-grant_write_unlock(d->grant_table);
+rc = gnttab_map_frame(d, idx, gfn, );
+if ( rc )
+return rc;
 
-if ( mfn_eq(mfn, INVALID_MFN) )
-return -EINVAL;
+t = p2m_ram_rw;
 
 break;
 case XENMAPSPACE_shared_info:
diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index e4fa60e928..08e8909270 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -4680,40 +4680,21 @@ int xenmem_add_to_physmap_one(
 {
 struct page_info *page = NULL;
 unsigned long gfn = 0; /* gcc ... */
-unsigned long prev_mfn, mfn = 0, old_gpfn;
+unsigned long prev_mfn, old_gpfn;
 int rc = 0;
+mfn_t mfn = INVALID_MFN;
 p2m_type_t p2mt;
 
 switch ( space )
 {
 case XENMAPSPACE_shared_info:
 if ( idx == 0 )
-mfn = virt_to_mfn(d->shared_info);
+mfn = _mfn(virt_to_mfn(d->shared_info));
 break;
 case XENMAPSPACE_grant_table:
-grant_write_lock(d->grant_table);
-
-if ( d->grant_table->gt_version == 0 )
-d->grant_table->gt_version = 1;
-
-if ( d->grant_table->gt_version == 2 &&
- (idx & XENMAPIDX_grant_table_status) )
-{
-idx &= ~XENMAPIDX_grant_table_status;
-if ( idx < nr_status_frames(d->grant_table) )
-mfn = virt_to_mfn(d->grant_table->status[idx]);
-}
-else
-{
-if ( (idx >= nr_grant_frames(d->grant_table)) &&
- (idx < max_grant_frames) )
-gnttab_grow_table(d, idx + 1);
-
-if ( idx < nr_grant_frames(d->grant_table) )
-mfn = virt_to_mfn(d->grant_table->shared_raw[idx]);
-}
-
-grant_write_unlock(d->grant_table);
+rc = gnttab_map_frame(d, idx, gpfn, );
+if ( rc )
+return rc;
 break;
 case XENMAPSPACE_gmfn_range:
 case XENMAPSPACE_gmfn:
@@ -4730,8 +4711,8 @@ int xenmem_add_to_physmap_one(
 }
 if ( !get_page_from_mfn(_mfn(idx), d) )
 break;
-mfn = idx;
-page = mfn_to_page(_mfn(mfn));
+mfn = _mfn(idx);
+page = mfn_to_page(mfn);
 break;
 }
 case XENMAPSPACE_gmfn_foreign:
@@ -4740,7 +4721,7 @@ int xenmem_add_to_physmap_one(
 break;
 }
 
-if ( !paging_mode_translate(d) || (mfn == 0) )
+if ( !paging_mode_translate(d) || mfn_eq(mfn, INVALID_MFN) )
 {
 rc = -EINVAL;
 goto put_both;
@@ -4764,16 +4745,16 @@ int xenmem_add_to_physmap_one(
 goto put_both;
 
 /* Unmap from old location, if any. */
-old_gpfn = get_gpfn_from_mfn(mfn);
+old_gpfn = get_gpfn_from_mfn(mfn_x(mfn));
 ASSERT( old_gpfn != SHARED_M2P_ENTRY 

[Xen-devel] [PATCH v6 06/12] tools: set grant limits for xenstore stubdom

[Juergen Gross]

When creating a Xenstore stubdom set the grant limits: the stubdom
will need to setup a very limited amount of grants only, so 1 grant
frame is enough. For being able to support up to 32768 domains it
will need 128 maptrack frames (1 mapping per domain, 256 maptrack
entries per maptrack frame).

Signed-off-by: Juergen Gross 
---
 tools/helpers/init-xenstore-domain.c | 11 +++
 1 file changed, 11 insertions(+)

diff --git a/tools/helpers/init-xenstore-domain.c 
b/tools/helpers/init-xenstore-domain.c
index 8a41ee7d3a..047ad0cb1d 100644
--- a/tools/helpers/init-xenstore-domain.c
+++ b/tools/helpers/init-xenstore-domain.c
@@ -105,6 +105,17 @@ static int build(xc_interface *xch)
 fprintf(stderr, "xc_domain_setmaxmem failed\n");
 goto err;
 }
+/*
+ * 1 grant frame is enough: we don't need many grants.
+ * Mini-OS doesn't like less than 4, though, so use 4.
+ * 128 maptrack frames: 256 entries per frame, enough for 32768 domains.
+ */
+rv = xc_domain_set_gnttab_limits(xch, domid, 4, 128);
+if ( rv )
+{
+fprintf(stderr, "xc_domain_set_gnttab_limits failed\n");
+goto err;
+}
 rv = xc_domain_set_memmap_limit(xch, domid, limit_kb);
 if ( rv )
 {
-- 
2.12.3


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v6 11/12] xen: make grant resource limits per domain

[Juergen Gross]

Instead of using the same global resource limits of grant tables (max.
number of grant frames, max. number of maptrack frames) for all domains
make these limits per domain. This will allow setting individual limits
in the future. For now initialize the per domain limits with the global
values.

Signed-off-by: Juergen Gross 
---
V6:
- several changes due to new patch order

V3:
- correct error message (Paul Durrant)
---
 xen/common/compat/grant_table.c   |  31 +++---
 xen/common/grant_table.c  | 121 --
 xen/include/asm-arm/grant_table.h |   6 +-
 3 files changed, 88 insertions(+), 70 deletions(-)

diff --git a/xen/common/compat/grant_table.c b/xen/common/compat/grant_table.c
index cce3ff0b9a..ff1d678f01 100644
--- a/xen/common/compat/grant_table.c
+++ b/xen/common/compat/grant_table.c
@@ -157,21 +157,14 @@ int compat_grant_table_op(unsigned int cmd,
 unsigned int max_frame_list_size_in_page =
 (COMPAT_ARG_XLAT_SIZE - sizeof(*nat.setup)) /
 sizeof(*nat.setup->frame_list.p);
-if ( max_frame_list_size_in_page < max_grant_frames )
-{
-gdprintk(XENLOG_WARNING,
- "max_grant_frames is too large (%u,%u)\n",
- max_grant_frames, max_frame_list_size_in_page);
-rc = -EINVAL;
-}
-else
-{
+
 #define XLAT_gnttab_setup_table_HNDL_frame_list(_d_, _s_) \
-set_xen_guest_handle((_d_)->frame_list, (unsigned long 
*)(nat.setup + 1))
-XLAT_gnttab_setup_table(nat.setup, );
+set_xen_guest_handle((_d_)->frame_list, (unsigned long 
*)(nat.setup + 1))
+XLAT_gnttab_setup_table(nat.setup, );
 #undef XLAT_gnttab_setup_table_HNDL_frame_list
-rc = gnttab_setup_table(guest_handle_cast(nat.uop, 
gnttab_setup_table_t), 1);
-}
+rc = gnttab_setup_table(guest_handle_cast(nat.uop,
+  
gnttab_setup_table_t),
+1, max_frame_list_size_in_page);
 }
 ASSERT(rc <= 0);
 if ( rc == 0 )
@@ -294,16 +287,6 @@ int compat_grant_table_op(unsigned int cmd,
 rc = -EFAULT;
 break;
 }
-if ( max_frame_list_size_in_pages <
- grant_to_status_frames(max_grant_frames) )
-{
-gdprintk(XENLOG_WARNING,
- "grant_to_status_frames(max_grant_frames) is too 
large (%u,%u)\n",
- grant_to_status_frames(max_grant_frames),
- max_frame_list_size_in_pages);
-rc = -EINVAL;
-break;
-}
 
 #define XLAT_gnttab_get_status_frames_HNDL_frame_list(_d_, _s_) \
 set_xen_guest_handle((_d_)->frame_list, (uint64_t 
*)(nat.get_status + 1))
@@ -312,7 +295,7 @@ int compat_grant_table_op(unsigned int cmd,
 
 rc = gnttab_get_status_frames(
 guest_handle_cast(nat.uop, gnttab_get_status_frames_t),
-count);
+count, max_frame_list_size_in_pages);
 if ( rc >= 0 )
 {
 #define XLAT_gnttab_get_status_frames_HNDL_frame_list(_d_, _s_) \
diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index 64e78e31f6..dc878a84f9 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -55,6 +55,9 @@ struct grant_table {
  * what version to use yet.
  */
 unsigned int  gt_version;
+/* Resource limits of the domain. */
+unsigned int  max_grant_frames;
+unsigned int  max_maptrack_frames;
 /* Table size. Number of frames shared with guest */
 unsigned int  nr_grant_frames;
 /* Number of grant status frames shared with guest (for version 2) */
@@ -291,8 +294,8 @@ num_act_frames_from_sha_frames(const unsigned int num)
 return DIV_ROUND_UP(num * sha_per_page, ACGNT_PER_PAGE);
 }
 
-#define max_nr_active_grant_frames \
-num_act_frames_from_sha_frames(max_grant_frames)
+#define max_nr_active_grant_frames(gt) \
+num_act_frames_from_sha_frames(gt->max_grant_frames)
 
 static inline unsigned int
 nr_active_grant_frames(struct grant_table *gt)
@@ -529,7 +532,7 @@ get_maptrack_handle(
  * out of memory, try stealing an entry from another VCPU (in case the
  * guest isn't mapping across its VCPUs evenly).
  */
-if ( nr_maptrack_frames(lgt) < max_maptrack_frames )
+if ( nr_maptrack_frames(lgt) < lgt->max_maptrack_frames )
 new_mt = alloc_xenheap_page();
 
 if ( !new_mt )
@@ -1668,23 +1671,26 @@ grant_table_init(struct grant_table *gt)
 
 /* Active grant table. */
 gt->active = xzalloc_array(struct active_grant_entry *,
-

[Xen-devel] [PATCH v6 05/12] libxc: add libxc support for setting grant table resource limits

[Juergen Gross]

Add a new libxc function xc_domain_set_gnttbl_limits() setting the
limits for the maximum numbers of grant table frames and maptrack
frames of a domain.

Signed-off-by: Juergen Gross 
Reviewed-by: Paul Durrant 
Acked-by: Wei Liu 
---
V4:
- use domid_t (Wei Liu)
---
 tools/libxc/include/xenctrl.h | 14 ++
 tools/libxc/xc_domain.c   | 13 +
 2 files changed, 27 insertions(+)

diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 43151cb415..ab34fb4f70 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -1064,6 +1064,20 @@ int xc_domain_set_virq_handler(xc_interface *xch, 
uint32_t domid, int virq);
 int xc_domain_set_max_evtchn(xc_interface *xch, uint32_t domid,
  uint32_t max_port);
 
+/**
+ * Set the maximum number of grant frames and/or maptrack frames a domain
+ * can have. Can only be used at domain setup time. A zero value means
+ * no change.
+ *
+ * @param xch a handle to an open hypervisor interface
+ * @param domid the domain id
+ * @param grant_frames max. number of grant frames
+ * @param maptrack_frames max. number of maptrack frames
+ */
+int xc_domain_set_gnttab_limits(xc_interface *xch, domid_t domid,
+uint32_t grant_frames,
+uint32_t maptrack_frames);
+
 /*
  * CPUPOOL MANAGEMENT FUNCTIONS
  */
diff --git a/tools/libxc/xc_domain.c b/tools/libxc/xc_domain.c
index 3bab4e8bab..41b42d6637 100644
--- a/tools/libxc/xc_domain.c
+++ b/tools/libxc/xc_domain.c
@@ -2268,6 +2268,19 @@ int xc_domain_set_max_evtchn(xc_interface *xch, uint32_t 
domid,
 return do_domctl(xch, );
 }
 
+int xc_domain_set_gnttab_limits(xc_interface *xch, domid_t domid,
+uint32_t grant_frames,
+uint32_t maptrack_frames)
+{
+DECLARE_DOMCTL;
+
+domctl.cmd = XEN_DOMCTL_set_gnttab_limits;
+domctl.domain = domid;
+domctl.u.set_gnttab_limits.grant_frames = grant_frames;
+domctl.u.set_gnttab_limits.maptrack_frames = maptrack_frames;
+return do_domctl(xch, );
+}
+
 /* Plumbing Xen with vNUMA topology */
 int xc_domain_setvnuma(xc_interface *xch,
uint32_t domid,
-- 
2.12.3


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v6 00/12] xen: better grant v2 support

[Juergen Gross]

Currently Linux has no support for grant v2 as this would reduce the
maximum number of active grants by a factor of 2 compared to v1,
because the number of possible grants are limited by the allowed number
of grant frames and grant entries of v2 need twice as much bytes as
those of v1.

Unfortunately grant v2 is the only way to support either guests with
more than 16TB memory size or PV guests with memory above the 16TB
border, as grant v1 limits the frame number to be 32 bits wide.

In order to remove the disadvantage of grant v2 this patch series
adds support for setting per-domain values regarding grant limits.
Additionally the default limit of grant frames is doubled in case
of hosts with memory above the 16TB border.

Changes in V6:
- several new patches (1, 6, 7, 10, 12)
- order of patches re-arranged to support new hypercall now being
  mandatory
- lots of other small changes

Changes in V5:
- patch 6: add set_gnttab_limits to create_domain_common in xen.if
  (Daniel De Graaf)

Changes in V4:
- patch 3: make ret more local (Wei Liu)
- patch 7: use domid_t (Wei Liu)
- patch 8: rename configuration items to use max_ prefixes (Wei Liu)

Changes in V3:
- patch 1: update commit message
- patch 3: move call of grant_table_init() from gnttab_setup_table() to
  gnttab_grow_table() (Paul Durrant)
- patch 4: correct error message (Paul Durrant)
- patch 6: rename *gnttbl* to *gnttab* (Paul Durrant)

Changes in V2:
- add per-domain grant limits instead of different v1 and v2 limits
- double default limit for huge hosts

Juergen Gross (12):
  xen: correct gnttab_get_status_frames()
  xen: move XENMAPSPACE_grant_table code into grant_table.c
  xen: clean up grant_table.h
  xen: add new domctl hypercall to set grant table resource limits
  libxc: add libxc support for setting grant table resource limits
  tools: set grant limits for xenstore stubdom
  xl: add global grant limit config items
  libxl: add libxl support for setting grant table resource limits
  xen: delay allocation of grant table sub structures
  xen/arm: move arch specific grant table bits into grant_table.c
  xen: make grant resource limits per domain
  xen: make grant table limits boot parameters dom0 only

 docs/man/xl.cfg.pod.5.in |  16 ++
 docs/man/xl.conf.pod.5   |  12 ++
 tools/flask/policy/modules/dom0.te   |   2 +-
 tools/flask/policy/modules/xen.if|   2 +-
 tools/helpers/init-xenstore-domain.c |  11 +
 tools/libxc/include/xenctrl.h|  14 ++
 tools/libxc/xc_domain.c  |  13 ++
 tools/libxl/libxl.h  |  15 +-
 tools/libxl/libxl_dm.c   |   3 +
 tools/libxl/libxl_dom.c  |   6 +
 tools/libxl/libxl_mem.c  |   5 +
 tools/libxl/libxl_types.idl  |   3 +
 tools/xl/xl.c|  14 ++
 tools/xl/xl.h|   2 +
 tools/xl/xl_parse.c  |   9 +
 tools/xl/xl_sxp.c|   2 +
 xen/arch/arm/domain.c|   2 -
 xen/arch/arm/domain_build.c  |   2 +-
 xen/arch/arm/mm.c|  36 +---
 xen/arch/x86/mm.c|  43 ++--
 xen/common/compat/grant_table.c  |  31 +--
 xen/common/domctl.c  |   6 +
 xen/common/grant_table.c | 405 +--
 xen/include/asm-arm/domain.h |   1 -
 xen/include/asm-arm/grant_table.h|  23 +-
 xen/include/asm-x86/grant_table.h|   7 +
 xen/include/public/domctl.h  |  11 +
 xen/include/xen/grant_table.h|  94 +---
 xen/xsm/flask/hooks.c|   3 +
 xen/xsm/flask/policy/access_vectors  |   2 +
 30 files changed, 494 insertions(+), 301 deletions(-)

-- 
2.12.3


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v6 04/12] xen: add new domctl hypercall to set grant table resource limits

[Juergen Gross]

Add a domctl hypercall to set the domain's resource limits regarding
grant tables. It is accepted only as long as neither
gnttab_setup_table() has been called for the domain, nor the domain
has started to run.

Signed-off-by: Juergen Gross 
---
V6:
- moved earlier in series to support set_gnttab_limits being
  mandatory for domain creation

V5:
- add set_gnttab_limits to create_domain_common in xen.if
  (Daniel De Graaf)

V3:
- rename *gnttbl* to *gnttab* (Paul Durrant)
---
 tools/flask/policy/modules/dom0.te  |  2 +-
 tools/flask/policy/modules/xen.if   |  2 +-
 xen/common/domctl.c |  6 ++
 xen/common/grant_table.c| 19 +++
 xen/include/public/domctl.h | 11 +++
 xen/include/xen/grant_table.h   |  2 ++
 xen/xsm/flask/hooks.c   |  3 +++
 xen/xsm/flask/policy/access_vectors |  2 ++
 8 files changed, 45 insertions(+), 2 deletions(-)

diff --git a/tools/flask/policy/modules/dom0.te 
b/tools/flask/policy/modules/dom0.te
index 338caaf41e..1643b400f0 100644
--- a/tools/flask/policy/modules/dom0.te
+++ b/tools/flask/policy/modules/dom0.te
@@ -39,7 +39,7 @@ allow dom0_t dom0_t:domain {
 };
 allow dom0_t dom0_t:domain2 {
set_cpuid gettsc settsc setscheduler set_max_evtchn set_vnumainfo
-   get_vnumainfo psr_cmt_op psr_cat_op
+   get_vnumainfo psr_cmt_op psr_cat_op set_gnttab_limits
 };
 allow dom0_t dom0_t:resource { add remove };
 
diff --git a/tools/flask/policy/modules/xen.if 
b/tools/flask/policy/modules/xen.if
index 912640002e..55437496f6 100644
--- a/tools/flask/policy/modules/xen.if
+++ b/tools/flask/policy/modules/xen.if
@@ -52,7 +52,7 @@ define(`create_domain_common', `
settime setdomainhandle getvcpucontext set_misc_info };
allow $1 $2:domain2 { set_cpuid settsc setscheduler setclaim
set_max_evtchn set_vnumainfo get_vnumainfo cacheflush
-   psr_cmt_op psr_cat_op soft_reset };
+   psr_cmt_op psr_cat_op soft_reset set_gnttab_limits };
allow $1 $2:security check_context;
allow $1 $2:shadow enable;
allow $1 $2:mmu { map_read map_write adjust memorymap physmap pinpage 
mmuext_op updatemp };
diff --git a/xen/common/domctl.c b/xen/common/domctl.c
index 42658e5744..58381f8fe9 100644
--- a/xen/common/domctl.c
+++ b/xen/common/domctl.c
@@ -14,6 +14,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -1149,6 +1150,11 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) 
u_domctl)
 copyback = 1;
 break;
 
+case XEN_DOMCTL_set_gnttab_limits:
+ret = grant_table_set_limits(d, op->u.set_gnttab_limits.grant_frames,
+ op->u.set_gnttab_limits.maptrack_frames);
+break;
+
 default:
 ret = arch_do_domctl(op, d, u_domctl);
 break;
diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index 5b296f9134..28ee050bad 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -3640,6 +3640,25 @@ void grant_table_init_vcpu(struct vcpu *v)
 v->maptrack_tail = MAPTRACK_TAIL;
 }
 
+int grant_table_set_limits(struct domain *d, unsigned int grant_frames,
+   unsigned int maptrack_frames)
+{
+struct grant_table *gt = d->grant_table;
+int ret = -EBUSY;
+
+if ( !gt )
+return -ENOENT;
+
+grant_write_lock(gt);
+
+ret = 0;
+/* Set limits, alloc needed arrays. */
+
+grant_write_unlock(gt);
+
+return ret;
+}
+
 #ifdef CONFIG_HAS_MEM_SHARING
 int mem_sharing_gref_to_gfn(struct grant_table *gt, grant_ref_t ref,
 gfn_t *gfn, uint16_t *status)
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 50ff58f5b9..685c6ebc15 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -1163,6 +1163,15 @@ struct xen_domctl_psr_cat_op {
 typedef struct xen_domctl_psr_cat_op xen_domctl_psr_cat_op_t;
 DEFINE_XEN_GUEST_HANDLE(xen_domctl_psr_cat_op_t);
 
+struct xen_domctl_set_gnttab_limits {
+uint32_t grant_frames; /* IN: if 0, dont change */
+uint32_t maptrack_frames;  /* IN: if 0, dont change */
+};
+#if 0
+typedef struct xen_domctl_set_gnttab_limits xen_domctl_set_gnttab_limits_t;
+DEFINE_XEN_GUEST_HANDLE(xen_domctl_set_gnttab_limits_t);
+#endif
+
 struct xen_domctl {
 uint32_t cmd;
 #define XEN_DOMCTL_createdomain   1
@@ -1240,6 +1249,7 @@ struct xen_domctl {
 #define XEN_DOMCTL_monitor_op77
 #define XEN_DOMCTL_psr_cat_op78
 #define XEN_DOMCTL_soft_reset79
+#define XEN_DOMCTL_set_gnttab_limits 80
 #define XEN_DOMCTL_gdbsx_guestmemio1000
 #define XEN_DOMCTL_gdbsx_pausevcpu 1001
 #define XEN_DOMCTL_gdbsx_unpausevcpu   1002
@@ -1302,6 +1312,7 @@ struct xen_domctl {
 struct xen_domctl_psr_cmt_op

[Xen-devel] [PATCH v6 07/12] xl: add global grant limit config items

[Juergen Gross]

Add xl.conf config items for default values of grant limits:

max_grant_frames will set the default for the maximum number of grant
frames for a domain which will take effect if the domain's config file
doesn't specify a value. If max_grant_frames isn't set in xl.conf it
will default to 32 for hosts with all memory below 16TB and to 64 for
hosts with memory above 16TB.

max_maptrack_frames will set the default for the maximum number of
maptrack frames for a domain. If max_maptrack_frames isn't set in
xl.conf it will default to 0, as normally only backend domains need
maptrack frames.

Signed-off-by: Juergen Gross 
---
 docs/man/xl.conf.pod.5  | 12 
 tools/libxl/libxl.h |  9 -
 tools/libxl/libxl_mem.c |  5 +
 tools/xl/xl.c   | 14 ++
 tools/xl/xl.h   |  2 ++
 5 files changed, 41 insertions(+), 1 deletion(-)

diff --git a/docs/man/xl.conf.pod.5 b/docs/man/xl.conf.pod.5
index 88ab506609..fe2cf27ea4 100644
--- a/docs/man/xl.conf.pod.5
+++ b/docs/man/xl.conf.pod.5
@@ -77,6 +77,18 @@ operations (primarily domain creation).
 
 Default: C
 
+=item 

[Xen-devel] [PATCH v6 03/12] xen: clean up grant_table.h

[Juergen Gross]

Many definitions can be moved from xen/grant_table.h to
common/grant_table.c now, as they are no longer used in other sources.

Rearrange the elements of struct grant_table to minimize holes due to
alignment of elements.

Signed-off-by: Juergen Gross 
Reviewed-by: Paul Durrant 
Reviewed-by: Wei Liu 
---
V6:
- coding style (Jan Beulich)
- rearrange struct grant_table to minimize holes (Jan Beulich)
---
 xen/common/grant_table.c  | 83 --
 xen/include/xen/grant_table.h | 84 ---
 2 files changed, 81 insertions(+), 86 deletions(-)

diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index a462ea7905..5b296f9134 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -40,6 +40,45 @@
 #include 
 #include 
 
+/* Per-domain grant information. */
+struct grant_table {
+/*
+ * Lock protecting updates to grant table state (version, active
+ * entry list, etc.)
+ */
+percpu_rwlock_t   lock;
+/* Lock protecting the maptrack limit */
+spinlock_tmaptrack_lock;
+/*
+ * The defined versions are 1 and 2.  Set to 0 if we don't know
+ * what version to use yet.
+ */
+unsigned int  gt_version;
+/* Table size. Number of frames shared with guest */
+unsigned int  nr_grant_frames;
+/* Number of grant status frames shared with guest (for version 2) */
+unsigned int  nr_status_frames;
+/* Number of available maptrack entries. */
+unsigned int  maptrack_limit;
+/* Shared grant table (see include/public/grant_table.h). */
+union {
+void **shared_raw;
+struct grant_entry_v1 **shared_v1;
+union grant_entry_v2 **shared_v2;
+};
+/* State grant table (see include/public/grant_table.h). */
+grant_status_t   **status;
+/* Active grant table. */
+struct active_grant_entry **active;
+/* Mapping tracking table per vcpu. */
+struct grant_mapping **maptrack;
+};
+
+#ifndef DEFAULT_MAX_NR_GRANT_FRAMES /* to allow arch to override */
+/* Default maximum size of a grant table. [POLICY] */
+#define DEFAULT_MAX_NR_GRANT_FRAMES   32
+#endif
+
 unsigned int __read_mostly max_grant_frames;
 integer_param("gnttab_max_frames", max_grant_frames);
 
@@ -118,6 +157,18 @@ struct grant_mapping {
 uint32_t pad;   /* round size to a power of 2 */
 };
 
+/* Number of grant table frames. Caller must hold d's grant table lock. */
+static inline unsigned int nr_grant_frames(const struct grant_table *gt)
+{
+return gt->nr_grant_frames;
+}
+
+/* Number of status grant table frames. Caller must hold d's gr. table lock.*/
+static inline unsigned int nr_status_frames(const struct grant_table *gt)
+{
+return gt->nr_status_frames;
+}
+
 #define MAPTRACK_PER_PAGE (PAGE_SIZE / sizeof(struct grant_mapping))
 #define maptrack_entry(t, e) \
 ((t)->maptrack[(e)/MAPTRACK_PER_PAGE][(e)%MAPTRACK_PER_PAGE])
@@ -197,7 +248,27 @@ static inline void act_set_gfn(struct active_grant_entry 
*act, gfn_t gfn)
 #endif
 }
 
-DEFINE_PERCPU_RWLOCK_GLOBAL(grant_rwlock);
+static DEFINE_PERCPU_RWLOCK_GLOBAL(grant_rwlock);
+
+static inline void grant_read_lock(struct grant_table *gt)
+{
+percpu_read_lock(grant_rwlock, >lock);
+}
+
+static inline void grant_read_unlock(struct grant_table *gt)
+{
+percpu_read_unlock(grant_rwlock, >lock);
+}
+
+static inline void grant_write_lock(struct grant_table *gt)
+{
+percpu_write_lock(grant_rwlock, >lock);
+}
+
+static inline void grant_write_unlock(struct grant_table *gt)
+{
+percpu_write_unlock(grant_rwlock, >lock);
+}
 
 static inline void gnttab_flush_tlb(const struct domain *d)
 {
@@ -250,6 +321,14 @@ static inline void active_entry_release(struct 
active_grant_entry *act)
 spin_unlock(>lock);
 }
 
+#define GRANT_STATUS_PER_PAGE (PAGE_SIZE / sizeof(grant_status_t))
+#define GRANT_PER_PAGE (PAGE_SIZE / sizeof(grant_entry_v2_t))
+/* Number of grant table status entries. Caller must hold d's gr. table lock.*/
+static inline unsigned int grant_to_status_frames(unsigned int grant_frames)
+{
+return DIV_ROUND_UP(grant_frames * GRANT_PER_PAGE, GRANT_STATUS_PER_PAGE);
+}
+
 /* Check if the page has been paged out, or needs unsharing.
If rc == GNTST_okay, *page contains the page struct with a ref taken.
Caller must do put_page(*page).
@@ -1580,7 +1659,7 @@ gnttab_unpopulate_status_frames(struct domain *d, struct 
grant_table *gt)
  * Grow the grant table. The caller must hold the grant table's
  * write lock before calling this function.
  */
-int
+static int
 gnttab_grow_table(struct domain *d, unsigned int req_nr_frames)
 {
 struct grant_table *gt = d->grant_table;
diff --git a/xen/include/xen/grant_table.h b/xen/include/xen/grant_table.h
index 43ec6c4d80..43b07e60c5 100644
--- a/xen/include/xen/grant_table.h
+++ b/xen/include/xen/grant_table.h
@@ 

[Xen-devel] [PATCH v6 10/12] xen/arm: move arch specific grant table bits into grant_table.c

[Juergen Gross]

Instead of attaching the ARM specific grant table data to the domain
structure add it to struct grant_table. Add the needed arch functions
to the asm-*/grant_table.h includes.

Signed-off-by: Juergen Gross 
---
 xen/arch/arm/domain.c |  2 --
 xen/common/grant_table.c  | 27 ---
 xen/include/asm-arm/domain.h  |  1 -
 xen/include/asm-arm/grant_table.h | 26 +++---
 xen/include/asm-x86/grant_table.h | 12 +++-
 xen/include/xen/grant_table.h |  3 ++-
 6 files changed, 48 insertions(+), 23 deletions(-)

diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 6512f01463..ee4643f52a 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -486,13 +486,11 @@ struct domain *alloc_domain_struct(void)
 return NULL;
 
 clear_page(d);
-d->arch.grant_table_gfn = xzalloc_array(gfn_t, max_grant_frames);
 return d;
 }
 
 void free_domain_struct(struct domain *d)
 {
-xfree(d->arch.grant_table_gfn);
 free_xenheap_page(d);
 }
 
diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index bf6f32b27f..64e78e31f6 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -39,6 +39,7 @@
 #include 
 #include 
 #include 
+#include 
 
 /* Per-domain grant information. */
 struct grant_table {
@@ -72,6 +73,8 @@ struct grant_table {
 struct active_grant_entry **active;
 /* Mapping tracking table per vcpu. */
 struct grant_mapping **maptrack;
+
+struct grant_table_arch arch;
 };
 
 #ifndef DEFAULT_MAX_NR_GRANT_FRAMES /* to allow arch to override */
@@ -1658,6 +1661,8 @@ gnttab_unpopulate_status_frames(struct domain *d, struct 
grant_table *gt)
 static int
 grant_table_init(struct grant_table *gt)
 {
+int ret = -ENOMEM;
+
 if ( gt->active )
 return -EBUSY;
 
@@ -1665,34 +1670,40 @@ grant_table_init(struct grant_table *gt)
 gt->active = xzalloc_array(struct active_grant_entry *,
max_nr_active_grant_frames);
 if ( gt->active == NULL )
-goto no_mem;
+goto out;
 
 /* Tracking of mapped foreign frames table */
 gt->maptrack = vzalloc(max_maptrack_frames * sizeof(*gt->maptrack));
 if ( gt->maptrack == NULL )
-goto no_mem;
+goto out;
 
 /* Shared grant table. */
 gt->shared_raw = xzalloc_array(void *, max_grant_frames);
 if ( gt->shared_raw == NULL )
-goto no_mem;
+goto out;
 
 /* Status pages for grant table - for version 2 */
 gt->status = xzalloc_array(grant_status_t *,
grant_to_status_frames(max_grant_frames));
 if ( gt->status == NULL )
-goto no_mem;
+goto out;
+
+ret = gnttab_init_arch(gt);
+if ( ret )
+goto out;
 
 return 0;
 
- no_mem:
+ out:
+xfree(gt->status);
+gt->status = NULL;
 xfree(gt->shared_raw);
 gt->shared_raw = NULL;
 vfree(gt->maptrack);
 gt->maptrack = NULL;
 xfree(gt->active);
 gt->active = NULL;
-return -ENOMEM;
+return ret;
 }
 
 /*
@@ -3603,6 +3614,8 @@ grant_table_destroy(
 if ( t == NULL )
 return;
 
+gnttab_destroy_arch(t);
+
 for ( i = 0; i < nr_grant_frames(t); i++ )
 free_xenheap_page(t->shared_raw[i]);
 xfree(t->shared_raw);
@@ -3728,7 +3741,7 @@ int gnttab_map_frame(struct domain *d, unsigned long idx, 
gfn_t gfn,
 rc = -EINVAL;
 }
 
-gnttab_set_frame_gfn(d, idx, gfn);
+gnttab_set_frame_gfn(gt, idx, gfn);
 
 grant_write_unlock(gt);
 
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index 8dfc1d1ec2..a22a0f5101 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -50,7 +50,6 @@ struct arch_domain
 struct p2m_domain p2m;
 
 struct hvm_domain hvm_domain;
-gfn_t *grant_table_gfn;
 
 struct vmmio vmmio;
 
diff --git a/xen/include/asm-arm/grant_table.h 
b/xen/include/asm-arm/grant_table.h
index 0a248a765a..0870b5b782 100644
--- a/xen/include/asm-arm/grant_table.h
+++ b/xen/include/asm-arm/grant_table.h
@@ -6,6 +6,10 @@
 
 #define INITIAL_NR_GRANT_FRAMES 4
 
+struct grant_table_arch {
+gfn_t *gfn;
+};
+
 void gnttab_clear_flag(unsigned long nr, uint16_t *addr);
 int create_grant_host_mapping(unsigned long gpaddr,
 unsigned long mfn, unsigned int flags, unsigned int
@@ -22,11 +26,19 @@ static inline int replace_grant_supported(void)
 return 1;
 }
 
-static inline void gnttab_set_frame_gfn(struct domain *d, unsigned long idx,
-gfn_t gfn)
-{
-d->arch.grant_table_gfn[idx] = gfn;
-}
+#define gnttab_init_arch(gt) \
+( ((gt)->arch.gfn = xzalloc_array(gfn_t, max_grant_frames)) == 0 \
+  ? 0 : -ENOMEM )
+
+#define gnttab_destroy_arch(gt)  \
+do { \
+

[Xen-devel] [PATCH v6 08/12] libxl: add libxl support for setting grant table resource limits

[Juergen Gross]

Add new domain config items for setting the limits for the maximum
numbers of grant table frames and maptrack frames of a domain.

Signed-off-by: Juergen Gross 
---
V6:
- made set_gnttab_limits hypercall mandatory, taking defaults from
  xl.conf

V4:
- rename configuration items to use max_ prefixes (Wei Liu)
---
 docs/man/xl.cfg.pod.5.in| 16 
 tools/libxl/libxl.h |  6 ++
 tools/libxl/libxl_dm.c  |  3 +++
 tools/libxl/libxl_dom.c |  6 ++
 tools/libxl/libxl_types.idl |  3 +++
 tools/xl/xl_parse.c |  9 +
 tools/xl/xl_sxp.c   |  2 ++
 7 files changed, 45 insertions(+)

diff --git a/docs/man/xl.cfg.pod.5.in b/docs/man/xl.cfg.pod.5.in
index 79cb2eaea7..5b6d1f8e2f 100644
--- a/docs/man/xl.cfg.pod.5.in
+++ b/docs/man/xl.cfg.pod.5.in
@@ -444,6 +444,20 @@ unpausing the domain. With a properly constructed security 
policy (such
 as nomigrate_t in the example policy), this can be used to build a
 domain whose memory is not accessible to the toolstack domain.
 
+=item 

[Xen-devel] [PATCH v6 12/12] xen: make grant table limits boot parameters dom0 only

[Juergen Gross]

The boot parameters gnttab_max_frames and gnttab_max_maptrack_frames
are used for dom0 only now, as all other domains require a
XEN_DOMCTL_set_gnttab_limits call now. So make that explicit by
setting the boot values for dom0 only.

Signed-off-by: Juergen Gross 
---
 xen/arch/arm/domain_build.c   |  2 +-
 xen/common/grant_table.c  | 35 ++-
 xen/include/xen/grant_table.h |  4 +---
 3 files changed, 16 insertions(+), 25 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index d6f9585503..ecfeab0969 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -2097,7 +2097,7 @@ static void __init find_gnttab_region(struct domain *d,
 kinfo->gnttab_size = (_etext - _stext) & PAGE_MASK;
 
 /* Make sure the grant table will fit in the region */
-if ( (kinfo->gnttab_size >> PAGE_SHIFT) < max_grant_frames )
+if ( grant_table_verify_size(d, kinfo->gnttab_size >> PAGE_SHIFT) )
 panic("Cannot find a space for the grant table region\n");
 
 #ifdef CONFIG_ARM_32
diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index dc878a84f9..99eac11a16 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -85,21 +85,8 @@ struct grant_table {
 #define DEFAULT_MAX_NR_GRANT_FRAMES   32
 #endif
 
-unsigned int __read_mostly max_grant_frames;
-integer_param("gnttab_max_frames", max_grant_frames);
-
-/* The maximum number of grant mappings is defined as a multiplier of the
- * maximum number of grant table entries. This defines the multiplier used.
- * Pretty arbitrary. [POLICY]
- * As gnttab_max_nr_frames has been deprecated, this multiplier is deprecated 
too.
- * New options allow to set max_maptrack_frames and
- * map_grant_table_frames independently.
- */
 #define DEFAULT_MAX_MAPTRACK_FRAMES 1024
 
-static unsigned int __read_mostly max_maptrack_frames;
-integer_param("gnttab_max_maptrack_frames", max_maptrack_frames);
-
 /*
  * Note that the three values below are effectively part of the ABI, even if
  * we don't need to make them a formal part of it: A guest suspended for
@@ -3463,6 +3450,10 @@ grant_table_create(
 struct domain *d)
 {
 struct grant_table *t;
+static unsigned int max_grant_frames;
+static unsigned int max_maptrack_frames;
+integer_param("gnttab_max_frames", max_grant_frames);
+integer_param("gnttab_max_maptrack_frames", max_maptrack_frames);
 
 if ( (t = xzalloc(struct grant_table)) == NULL )
 return -ENOMEM;
@@ -3470,14 +3461,17 @@ grant_table_create(
 /* Simple stuff. */
 percpu_rwlock_resource_init(>lock, grant_rwlock);
 spin_lock_init(>maptrack_lock);
-t->max_grant_frames = max_grant_frames;
-t->max_maptrack_frames = max_maptrack_frames;
 
 /* Okay, install the structure. */
 d->grant_table = t;
 
 if ( d->domain_id == 0 )
+{
+t->max_grant_frames = max_grant_frames ? : DEFAULT_MAX_NR_GRANT_FRAMES;
+t->max_maptrack_frames =
+   max_maptrack_frames ? : DEFAULT_MAX_MAPTRACK_FRAMES;
 return grant_table_init(t);
+}
 
 return 0;
 }
@@ -3855,18 +3849,17 @@ static int __init gnttab_usage_init(void)
 {
 BUILD_BUG_ON(DEFAULT_MAX_MAPTRACK_FRAMES < DEFAULT_MAX_NR_GRANT_FRAMES);
 
-if ( !max_grant_frames )
-max_grant_frames = DEFAULT_MAX_NR_GRANT_FRAMES;
-
-if ( !max_maptrack_frames )
-max_maptrack_frames = DEFAULT_MAX_MAPTRACK_FRAMES;
-
 register_keyhandler('g', gnttab_usage_print_all,
 "print grant table usage", 1);
 return 0;
 }
 __initcall(gnttab_usage_init);
 
+bool __init grant_table_verify_size(struct domain *d, unsigned int frames)
+{
+return d->grant_table->max_grant_frames > frames;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/xen/grant_table.h b/xen/include/xen/grant_table.h
index f3f2fb9ebc..1633364ff3 100644
--- a/xen/include/xen/grant_table.h
+++ b/xen/include/xen/grant_table.h
@@ -30,9 +30,6 @@
 
 struct grant_table;
 
-/* The maximum size of a grant table. */
-extern unsigned int max_grant_frames;
-
 /* Create/destroy per-domain grant table context. */
 int grant_table_create(
 struct domain *d);
@@ -41,6 +38,7 @@ void grant_table_destroy(
 void grant_table_init_vcpu(struct vcpu *v);
 int grant_table_set_limits(struct domain *d, unsigned int grant_frames,
unsigned int maptrack_frames);
+bool grant_table_verify_size(struct domain *d, unsigned int frames);
 
 /*
  * Check if domain has active grants and log first 10 of them.
-- 
2.12.3


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v6 09/12] xen: delay allocation of grant table sub structures

[Juergen Gross]

Delay the allocation of the grant table sub structures in order to
allow modifying parameters needed for sizing of these structures at a
per domain basis. Either do it from gnttab_grow_table() or just
before the domain is started the first time.

Signed-off-by: Juergen Gross 
---
V6:
- move call of grant_table_init() for dom0 to grant_table_create()
  (Jan Beulich)
- move frame allocations to gnttab_grow_table() (Jan Beulich)
- several other changes due to new patch order

V4:
- make ret more local (Wei Liu)

V3:
- move call of grant_table_init() from gnttab_setup_table() to
  gnttab_grow_table() (Paul Durrant)
---
 xen/common/grant_table.c | 113 ++-
 1 file changed, 52 insertions(+), 61 deletions(-)

diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index 28ee050bad..bf6f32b27f 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -1655,6 +1655,46 @@ gnttab_unpopulate_status_frames(struct domain *d, struct 
grant_table *gt)
 gt->nr_status_frames = 0;
 }
 
+static int
+grant_table_init(struct grant_table *gt)
+{
+if ( gt->active )
+return -EBUSY;
+
+/* Active grant table. */
+gt->active = xzalloc_array(struct active_grant_entry *,
+   max_nr_active_grant_frames);
+if ( gt->active == NULL )
+goto no_mem;
+
+/* Tracking of mapped foreign frames table */
+gt->maptrack = vzalloc(max_maptrack_frames * sizeof(*gt->maptrack));
+if ( gt->maptrack == NULL )
+goto no_mem;
+
+/* Shared grant table. */
+gt->shared_raw = xzalloc_array(void *, max_grant_frames);
+if ( gt->shared_raw == NULL )
+goto no_mem;
+
+/* Status pages for grant table - for version 2 */
+gt->status = xzalloc_array(grant_status_t *,
+   grant_to_status_frames(max_grant_frames));
+if ( gt->status == NULL )
+goto no_mem;
+
+return 0;
+
+ no_mem:
+xfree(gt->shared_raw);
+gt->shared_raw = NULL;
+vfree(gt->maptrack);
+gt->maptrack = NULL;
+xfree(gt->active);
+gt->active = NULL;
+return -ENOMEM;
+}
+
 /*
  * Grow the grant table. The caller must hold the grant table's
  * write lock before calling this function.
@@ -1665,6 +1705,10 @@ gnttab_grow_table(struct domain *d, unsigned int 
req_nr_frames)
 struct grant_table *gt = d->grant_table;
 unsigned int i, j;
 
+ASSERT(gt->active);
+
+if ( req_nr_frames < INITIAL_NR_GRANT_FRAMES )
+req_nr_frames = INITIAL_NR_GRANT_FRAMES;
 ASSERT(req_nr_frames <= max_grant_frames);
 
 gdprintk(XENLOG_INFO,
@@ -3381,75 +3425,21 @@ grant_table_create(
 struct domain *d)
 {
 struct grant_table *t;
-unsigned int i, j;
 
 if ( (t = xzalloc(struct grant_table)) == NULL )
-goto no_mem_0;
+return -ENOMEM;
 
 /* Simple stuff. */
 percpu_rwlock_resource_init(>lock, grant_rwlock);
 spin_lock_init(>maptrack_lock);
-t->nr_grant_frames = INITIAL_NR_GRANT_FRAMES;
-
-/* Active grant table. */
-if ( (t->active = xzalloc_array(struct active_grant_entry *,
-max_nr_active_grant_frames)) == NULL )
-goto no_mem_1;
-for ( i = 0;
-  i < num_act_frames_from_sha_frames(INITIAL_NR_GRANT_FRAMES); i++ )
-{
-if ( (t->active[i] = alloc_xenheap_page()) == NULL )
-goto no_mem_2;
-clear_page(t->active[i]);
-for ( j = 0; j < ACGNT_PER_PAGE; j++ )
-spin_lock_init(>active[i][j].lock);
-}
-
-/* Tracking of mapped foreign frames table */
-t->maptrack = vzalloc(max_maptrack_frames * sizeof(*t->maptrack));
-if ( t->maptrack == NULL )
-goto no_mem_2;
-
-/* Shared grant table. */
-if ( (t->shared_raw = xzalloc_array(void *, max_grant_frames)) == NULL )
-goto no_mem_3;
-for ( i = 0; i < INITIAL_NR_GRANT_FRAMES; i++ )
-{
-if ( (t->shared_raw[i] = alloc_xenheap_page()) == NULL )
-goto no_mem_4;
-clear_page(t->shared_raw[i]);
-}
-
-/* Status pages for grant table - for version 2 */
-t->status = xzalloc_array(grant_status_t *,
-  grant_to_status_frames(max_grant_frames));
-if ( t->status == NULL )
-goto no_mem_4;
-
-for ( i = 0; i < INITIAL_NR_GRANT_FRAMES; i++ )
-gnttab_create_shared_page(d, t, i);
-
-t->nr_status_frames = 0;
 
 /* Okay, install the structure. */
 d->grant_table = t;
-return 0;
 
- no_mem_4:
-for ( i = 0; i < INITIAL_NR_GRANT_FRAMES; i++ )
-free_xenheap_page(t->shared_raw[i]);
-xfree(t->shared_raw);
- no_mem_3:
-vfree(t->maptrack);
- no_mem_2:
-for ( i = 0;
-  i < num_act_frames_from_sha_frames(INITIAL_NR_GRANT_FRAMES); i++ )
-free_xenheap_page(t->active[i]);
-xfree(t->active);
- no_mem_1:
-xfree(t);
- no_mem_0:
-return -ENOMEM;
+if ( d->domain_id == 0 )
+return 

[Xen-devel] [PATCH v6 01/12] xen: correct gnttab_get_status_frames()

[Juergen Gross]

In gnttab_get_status_frames() all accesses to nr_status_frames should
be done with the grant table lock held.

While at it correct coding style: labels should be indented by one
space.

Signed-off-by: Juergen Gross 
---
 xen/common/grant_table.c | 15 ---
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index c3895e6201..00ff075bd9 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -2866,19 +2866,19 @@ 
gnttab_get_status_frames(XEN_GUEST_HANDLE_PARAM(gnttab_get_status_frames_t) uop,
 
 gt = d->grant_table;
 
+op.status = GNTST_okay;
+
+grant_read_lock(gt);
+
 if ( unlikely(op.nr_frames > nr_status_frames(gt)) )
 {
 gdprintk(XENLOG_INFO, "Guest requested addresses for %d grant status "
  "frames, but only %d are available.\n",
  op.nr_frames, nr_status_frames(gt));
 op.status = GNTST_general_error;
-goto out2;
+goto unlock;
 }
 
-op.status = GNTST_okay;
-
-grant_read_lock(gt);
-
 for ( i = 0; i < op.nr_frames; i++ )
 {
 gmfn = gnttab_status_gmfn(d, gt, i);
@@ -2886,10 +2886,11 @@ 
gnttab_get_status_frames(XEN_GUEST_HANDLE_PARAM(gnttab_get_status_frames_t) uop,
 op.status = GNTST_bad_virt_addr;
 }
 
+ unlock:
 grant_read_unlock(gt);
-out2:
+ out2:
 rcu_unlock_domain(d);
-out1:
+ out1:
 if ( unlikely(__copy_field_to_guest(uop, , status)) )
 return -EFAULT;
 
-- 
2.12.3


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Booting signed xen.efi through shim

[Jan Beulich]

>>> On 13.09.17 at 16:40,  wrote:
> On Wed, Sep 13, 2017 at 3:21 AM, Jan Beulich  wrote:
> On 13.09.17 at 07:27,  wrote:
>>>Sections:
>>>Idx Name  Size  VMA   LMA   File off  
>>>Algn
>>>  0 .text 0017a1ba  82d08020  82d08020  1000  
>>> 2**12
>>>  CONTENTS, ALLOC, LOAD, CODE
>>>  1 .rodata   000826a0  82d08040  82d08040  0017c000  
>>> 2**5
>>>  CONTENTS, ALLOC, LOAD, DATA
>>>  2 .buildid  0035  82d0804826a0  82d0804826a0  001fe6a0  
>>> 2**2
>>>  CONTENTS, ALLOC, LOAD, READONLY, DATA
>>>  3 .init 00077df0  82d08060  82d08060  001ff000  
>>> 2**12
>>>  CONTENTS, ALLOC, LOAD, CODE, DATA
>>>  4 .data.re  aa40  82d08080  82d08080  00277000  
>>> 2**7
>>>  CONTENTS, ALLOC, LOAD, DATA
>>>  5 .data 000105a8  82d08080b000  82d08080b000  00282000  
>>> 2**12
>>>  CONTENTS, ALLOC, LOAD, DATA
>>>  6 .bss  00143280  82d08082  82d08082    
>>> 2**4
>>>  ALLOC, RELOC
>>
>> Objdump is apparently ignoring a section attribute bit here - my
>> own utility properly prints "bss" in addition to "read" (which presumably
>> matches "ALLOC" above, albeit that's a bogus translation apparently
>> applying ELF semantics to COFF). You'll want to check that bit 7 in the
>> section attributes is set. I'm also puzzled by "RELOC", but I do see a
>> matching bit dumped here; not sure why that's being set.
> 
> Looking at it with readpe I get:
> 
> Name:.bss
> Virtual Address: 0x82
> Physical Address:0x143280
> Size:0 (0 bytes)
> Pointer To Data: 0
> Relocations: 0
> Characteristics: 0xc180
>  contains uninitialized data
>  contains extended relocations
>  is readable
>  is writable
> 
> So bit 7 is set AFAICT.

Good.

>> It is certainly the case that .bss style sections are expected to have a
>> zero file offset, as there's no data for such sections inside the file (note
>> the missing "CONTENTS" above. So I would conclude that, unless the
>> bss flag really got lost, it's a shim loader bug. Since other people can
>> load xen.efi with the shim, that might be a problem with the particular
>> version you're using.
> 
> Perhaps, I'm using the latest master
> (e22a7b5b772dba6588dd955dc017e572f7e29784) from
> https://github.com/mjg59/shim, the one being linked to on the wiki. If
> there is a known good version, I would be happy to give that a shot
> and see if I can get it working.

I have no idea. What I'd suggest you to try is to zap that stray
"contains extended relocations" flag. I've written down to go hunt
for where it comes from, but I don't have the time to do that right
away.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] x86/oprofile: Add a missing space to initialisation failure message

[Jan Beulich]

>>> On 13.09.17 at 15:41,  wrote:
> Signed-off-by: Andrew Cooper 

Acked-by: Jan Beulich 



___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 03/17] x86emul: build SIMD tests with -Os

[Jan Beulich]

>>> On 13.09.17 at 17:19,  wrote:
> On Wed, Jun 21, 2017 at 1:00 PM, Jan Beulich  wrote:
>> Namely in the context of putting together subsequent patches I've
>> noticed that together with the touch() macro using -Os further
>> increases the chances of the compiler using memory operands for the
>> instructions we actually care to test.
>>
>> Signed-off-by: Jan Beulich 
> 
> I'm not sure how I'm supposed to evaluate the claim, but it certainly

Well, looking at the generated code for those tests is probably the
only way.

> looks like the patch does what it does on the tin, so:
> 
> Reviewed-by: George Dunlap 

Thanks (also for the other patch)!

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 01/17] x86emul: support remaining AVX insns

[Jan Beulich]

>>> On 13.09.17 at 17:02,  wrote:
> On Wed, Jun 21, 2017 at 12:59 PM, Jan Beulich  wrote:
>> I.e. those not being equivalents of SSEn ones.
>>
>> There's one necessary change to generic code: Faulting behavior of
>> VMASKMOVP{S,D} requires us to do partial reads/writes.
>>
>> Signed-off-by: Jan Beulich 
> 
> Having dug a bit through the manuals about AVX, then come back to this
> patch again, I feel the same way that I did when I first read it:
> This discription is woefully inadequate.  It doesn't look so much like
> a changelog as a note to yourself.  If someone else sent a patch
> description like this for a patch this complicated and expected you to
> review it you'd certainly complain, and there's a good chance you
> wouldn't review it.
> 
> It's not your job to teach me how this code is laid out and how it
> works, but there at least needs to be enough description of what the
> patch is actually trying to do that I have some hope of reconstructing
> its intent.

Hmm, for someone not familiar with prior changes in this direction
(SSE etc) I can see how the brevity might be confusing / unhelpful.
However, the title of the patch says all that is to be said about
"its intent". The description really just points out the one thing
that is not in line with code that's already there. And I'm sure you
don't mean me to enumerate the individual instructions support is
being added for. Yet beyond that and beyond the partial write
handling there's really nothing the patch does. So I'm sort of lost
as to what additional information I could provide.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 02/17] x86emul: re-order cases of main switch statement

[George Dunlap]

On Wed, Jun 21, 2017 at 12:59 PM, Jan Beulich  wrote:
> Re-store intended numerical ordering, which has become "violated"
> mostly by incremental additions where moving around bigger chunks did
> not seem advisable. One exception though at the very top of the
> switch(): Keeping the arithmetic ops together seems preferable over
> entirely strict ordering.

+1

> Additionally move a few macro definitions before their first uses (the
> placement is benign as long as those uses are themselves only macro
> definitions, but that's going to change when those macros have helpers
> broken out).
>
> No (intended) functional change.
>
> Signed-off-by: Jan Beulich 

As far as I can tell, the 'no functional change' is true:

Reviewed-by:  George Dunlap 

>
> --- a/xen/arch/x86/x86_emulate/x86_emulate.c
> +++ b/xen/arch/x86/x86_emulate/x86_emulate.c
> @@ -843,6 +843,27 @@ do{ asm volatile (
>  #define __emulate_1op_8byte(_op, _dst, _eflags)
>  #endif /* __i386__ */
>
> +#define fail_if(p)  \
> +do {\
> +rc = (p) ? X86EMUL_UNHANDLEABLE : X86EMUL_OKAY; \
> +if ( rc ) goto done;\
> +} while (0)
> +
> +static inline int mkec(uint8_t e, int32_t ec, ...)
> +{
> +return (e < 32 && ((1u << e) & EXC_HAS_EC)) ? ec : X86_EVENT_NO_EC;
> +}
> +
> +#define generate_exception_if(p, e, ec...)\
> +({  if ( (p) ) {  \
> +x86_emul_hw_exception(e, mkec(e, ##ec, 0), ctxt); \
> +rc = X86EMUL_EXCEPTION;   \
> +goto done;\
> +} \
> +})
> +
> +#define generate_exception(e, ec...) generate_exception_if(true, e, ##ec)
> +
>  #ifdef __XEN__
>  # define invoke_stub(pre, post, constraints...) do {\
>  union stub_exception_token res_ = { .raw = ~0 };\
> @@ -911,27 +932,6 @@ do{ asm volatile (
>  # define mode_64bit() false
>  #endif
>
> -#define fail_if(p)  \
> -do {\
> -rc = (p) ? X86EMUL_UNHANDLEABLE : X86EMUL_OKAY; \
> -if ( rc ) goto done;\
> -} while (0)
> -
> -static inline int mkec(uint8_t e, int32_t ec, ...)
> -{
> -return (e < 32 && ((1u << e) & EXC_HAS_EC)) ? ec : X86_EVENT_NO_EC;
> -}
> -
> -#define generate_exception_if(p, e, ec...)\
> -({  if ( (p) ) {  \
> -x86_emul_hw_exception(e, mkec(e, ##ec, 0), ctxt); \
> -rc = X86EMUL_EXCEPTION;   \
> -goto done;\
> -} \
> -})
> -
> -#define generate_exception(e, ec...) generate_exception_if(true, e, ##ec)
> -
>  /*
>   * Given byte has even parity (even number of 1s)? SDM Vol. 1 Sec. 3.4.3.1,
>   * "Status Flags": EFLAGS.PF reflects parity of least-sig. byte of result 
> only.
> @@ -3596,6 +3596,11 @@ x86_emulate(
>  dst.bytes = 2;
>  break;
>
> +case 0x8d: /* lea */
> +generate_exception_if(ea.type != OP_MEM, EXC_UD);
> +dst.val = ea.mem.off;
> +break;
> +
>  case 0x8e: /* mov r/m,Sreg */
>  seg = modrm_reg & 7; /* REX.R is ignored. */
>  generate_exception_if(!is_x86_user_segment(seg) ||
> @@ -3607,11 +3612,6 @@ x86_emulate(
>  dst.type = OP_NONE;
>  break;
>
> -case 0x8d: /* lea */
> -generate_exception_if(ea.type != OP_MEM, EXC_UD);
> -dst.val = ea.mem.off;
> -break;
> -
>  case 0x8f: /* pop (sole member of Grp1a) */
>  generate_exception_if((modrm_reg & 7) != 0, EXC_UD);
>  /* 64-bit mode: POP defaults to a 64-bit operand. */
> @@ -5746,12 +5746,6 @@ x86_emulate(
>  _regs.r(ax) = (uint32_t)msr_val;
>  break;
>
> -case X86EMUL_OPC(0x0f, 0x40) ... X86EMUL_OPC(0x0f, 0x4f): /* cmovcc */
> -vcpu_must_have(cmov);
> -if ( test_cc(b, _regs.eflags) )
> -dst.val = src.val;
> -break;
> -
>  case X86EMUL_OPC(0x0f, 0x34): /* sysenter */
>  vcpu_must_have(sep);
>  generate_exception_if(mode_ring0(), EXC_GP, 0);
> @@ -5834,6 +5828,12 @@ x86_emulate(
>  singlestep = _regs.eflags & X86_EFLAGS_TF;
>  break;
>
> +case X86EMUL_OPC(0x0f, 0x40) ... X86EMUL_OPC(0x0f, 0x4f): /* cmovcc */
> +vcpu_must_have(cmov);
> +if ( test_cc(b, _regs.eflags) )
> +dst.val = src.val;
> +break;
> +
>  

Re: [Xen-devel] [PATCH 03/17] x86emul: build SIMD tests with -Os

[George Dunlap]

On Wed, Jun 21, 2017 at 1:00 PM, Jan Beulich  wrote:
> Namely in the context of putting together subsequent patches I've
> noticed that together with the touch() macro using -Os further
> increases the chances of the compiler using memory operands for the
> instructions we actually care to test.
>
> Signed-off-by: Jan Beulich 

I'm not sure how I'm supposed to evaluate the claim, but it certainly
looks like the patch does what it does on the tin, so:

Reviewed-by: George Dunlap 

>
> --- a/tools/tests/x86_emulator/Makefile
> +++ b/tools/tests/x86_emulator/Makefile
> @@ -45,17 +45,17 @@ define simd-defs
>  $(1)-cflags := \
> $(foreach vec,$($(1)-vecs), \
>   $(foreach int,$($(1)-ints), \
> -   "-D_$(vec)i$(int) -m$(1) $(call non-sse,$(1)) -O2 
> -DVEC_SIZE=$(vec) -DINT_SIZE=$(int)" \
> -   "-D_$(vec)u$(int) -m$(1) $(call non-sse,$(1)) -O2 
> -DVEC_SIZE=$(vec) -DUINT_SIZE=$(int)") \
> +   "-D_$(vec)i$(int) -m$(1) $(call non-sse,$(1)) -Os 
> -DVEC_SIZE=$(vec) -DINT_SIZE=$(int)" \
> +   "-D_$(vec)u$(int) -m$(1) $(call non-sse,$(1)) -Os 
> -DVEC_SIZE=$(vec) -DUINT_SIZE=$(int)") \
>   $(foreach flt,$($(1)-flts), \
> -   "-D_$(vec)f$(flt) -m$(1) $(call non-sse,$(1)) -O2 
> -DVEC_SIZE=$(vec) -DFLOAT_SIZE=$(flt)")) \
> +   "-D_$(vec)f$(flt) -m$(1) $(call non-sse,$(1)) -Os 
> -DVEC_SIZE=$(vec) -DFLOAT_SIZE=$(flt)")) \
> $(foreach flt,$($(1)-flts), \
> - "-D_f$(flt) -m$(1) $(call non-sse,$(1)) -mfpmath=sse -O2 
> -DFLOAT_SIZE=$(flt)")
> + "-D_f$(flt) -m$(1) $(call non-sse,$(1)) -mfpmath=sse -Os 
> -DFLOAT_SIZE=$(flt)")
>  $(1)-avx-cflags := \
> $(foreach vec,$($(1)-vecs), \
>   $(foreach int,$($(1)-ints), \
> -   "-D_$(vec)i$(int) -m$(1) $(sse2avx-$(1)) -O2 -DVEC_SIZE=$(vec) 
> -DINT_SIZE=$(int)" \
> -   "-D_$(vec)u$(int) -m$(1) $(sse2avx-$(1)) -O2 -DVEC_SIZE=$(vec) 
> -DUINT_SIZE=$(int)"))
> +   "-D_$(vec)i$(int) -m$(1) $(sse2avx-$(1)) -Os -DVEC_SIZE=$(vec) 
> -DINT_SIZE=$(int)" \
> +   "-D_$(vec)u$(int) -m$(1) $(sse2avx-$(1)) -Os -DVEC_SIZE=$(vec) 
> -DUINT_SIZE=$(int)"))
>  endef
>
>  $(foreach flavor,$(SIMD),$(eval $(call simd-defs,$(flavor
>
>
>
>
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel
>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel