[Xen-devel] [BUG] panic: "IO-APIC + timer doesn't work" - several people have reproduced

2019-12-30 Thread Aaron Janse
Hello all, After attempting to install QubesOS on a new laptop, I've stumbled upon a group of people with an assortment of laptops but all the same problem: a Xen panic stating "IO-APIC + timer doesn't work!" Many of us are on different stages of debugging this, so I'll cite all our efforts

[Xen-devel] [xen-unstable test] 145407: regressions - FAIL

2019-12-30 Thread osstest service owner
flight 145407 xen-unstable real [real] http://logs.test-lab.xenproject.org/osstest/logs/145407/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-amd64-qemuu-nested-intel 17 debian-hvm-install/l1/l2 fail REGR. vs. 145025 Tests

Re: [Xen-devel] REGRESSION: Xen 4.13 RC5 fails to bootstrap Dom0 on ARM

2019-12-30 Thread Roman Shaposhnik
Hi Julien, On Sun, Dec 29, 2019 at 10:01 AM Julien Grall wrote: > > Hi, > > On 21/12/2019 01:37, Roman Shaposhnik wrote: > > On Thu, Dec 19, 2019 at 4:01 PM Stefano Stabellini > > wrote: > >> > >> On Thu, 19 Dec 2019, Julien Grall wrote: > > In fact most of people on Arm are using GRUB

[Xen-devel] [qemu-mainline test] 145401: regressions - FAIL

2019-12-30 Thread osstest service owner
flight 145401 qemu-mainline real [real] http://logs.test-lab.xenproject.org/osstest/logs/145401/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-i386-freebsd10-i386 14 guest-saverestore fail REGR. vs. 144861

Re: [Xen-devel] [PATCH v2 00/20] VM forking

2019-12-30 Thread Tamas K Lengyel
On Mon, Dec 30, 2019 at 5:20 PM Julien Grall wrote: > > Hi, > > On Mon, 30 Dec 2019, 20:49 Tamas K Lengyel, wrote: >> >> On Mon, Dec 30, 2019 at 11:43 AM Julien Grall wrote: >> But keep in mind that the "fork-vm" command even with this update >> would still not produce for you a "fully

Re: [Xen-devel] [PATCH v2 00/20] VM forking

2019-12-30 Thread Julien Grall
Hi, On Mon, 30 Dec 2019, 20:49 Tamas K Lengyel, wrote: > On Mon, Dec 30, 2019 at 11:43 AM Julien Grall wrote: > But keep in mind that the "fork-vm" command even with this update > would still not produce for you a "fully functional" VM on its own. > The user still has to produce a new VM

Re: [Xen-devel] [PATCH v2 00/20] VM forking

2019-12-30 Thread Tamas K Lengyel
On Mon, Dec 30, 2019 at 11:43 AM Julien Grall wrote: > > Hi Tamas, > > On 30/12/2019 18:15, Tamas K Lengyel wrote: > > On Mon, Dec 30, 2019 at 10:59 AM Roger Pau Monné > > wrote: > >> > >> On Thu, Dec 19, 2019 at 08:58:01AM -0700, Tamas K Lengyel wrote: > >>> On Thu, Dec 19, 2019 at 2:48 AM

[Xen-devel] [xen-unstable test] 145393: regressions - FAIL

2019-12-30 Thread osstest service owner
flight 145393 xen-unstable real [real] http://logs.test-lab.xenproject.org/osstest/logs/145393/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-amd64-qemuu-nested-intel 17 debian-hvm-install/l1/l2 fail REGR. vs. 145025 Tests

[Xen-devel] [PATCH v4 6/7] Add guide on Communication Best Practice

2019-12-30 Thread Lars Kurth
From: Lars Kurth This guide covers the bulk on Best Practice related to code review It primarily focusses on code review interactions It also covers how to deal with Misunderstandings and Cultural Differences Changes since v3 * Fixed typo Changes since v2 (added in v2) * Fix typos * Extended

[Xen-devel] [PATCH v4 4/7] Add Communication Guide

2019-12-30 Thread Lars Kurth
From: Lars Kurth This document is a portal page that lays out our gold standard, best practices for some common situations and mechanisms to help resolve issues that can have a negative effect on our community. Detail is covered in subsequent documents Changes since v3 - Also changes the TODO

[Xen-devel] [PATCH v4 0/7] Code of Conduct + Extra Guides and Best Practices + VOTE

2019-12-30 Thread Lars Kurth
From: Lars Kurth This series proposes a concrete version of the Xen Project CoC based on v1.4 of the Contributor Covenant. See [1] Closing the discussion == I think we are at the point where we are ready to publish our guidance. Feedback has been minor since the last version

[Xen-devel] [PATCH v4 5/7] Add Code Review Guide

2019-12-30 Thread Lars Kurth
From: Lars Kurth This document highlights what reviewers such as maintainers and committers look for when reviewing code. It sets expectations for code authors and provides a framework for code reviewers. Changes since v3 * Added example under *Workflow from a Reviewer's Perspective* section *

[Xen-devel] [PATCH v4 1/7] Import v1.4 of Contributor Covenant CoC

2019-12-30 Thread Lars Kurth
From: Lars Kurth Signed-off-by: Lars Kurth --- Cc: minios-de...@lists.xenproject.org Cc: xen-...@lists.xenproject.org Cc: win-pv-de...@lists.xenproject.org Cc: mirageos-de...@lists.xenproject.org Cc: committ...@xenproject.org --- code-of-conduct.md | 76

[Xen-devel] [PATCH v4 3/7] Reformat Xen Project CoC to fit into 80 character limit

2019-12-30 Thread Lars Kurth
From: Lars Kurth No content changes Signed-off-by: Lars Kurth --- Cc: minios-de...@lists.xenproject.org Cc: xen-...@lists.xenproject.org Cc: win-pv-de...@lists.xenproject.org Cc: mirageos-de...@lists.xenproject.org Cc: committ...@xenproject.org --- code-of-conduct.md | 62

[Xen-devel] [PATCH v4 7/7] Added Resolving Disagreement

2019-12-30 Thread Lars Kurth
From: Lars Kurth This guide provides Best Practice on identifying and resolving common classes of disagreement Changes since v3 * Fixed broken http link (typo) Changes since v2 (added in v2) * Fix typos * Add section: "Issue: Multiple ways to solve a problem" * Changed line wrapping to 80

[Xen-devel] [PATCH v4 2/7] Xen Project Code of Conduct

2019-12-30 Thread Lars Kurth
From: Lars Kurth Specific changes to the baseline: * Replace list of positive behaviors with link to separate process * Replace maintainers with project leadership (except in our pledge where maintainers is more appropriate) * Add 'of all sub-projects' to clarify scope of CoC * Rename

Re: [Xen-devel] [PATCH v2 00/20] VM forking

2019-12-30 Thread Julien Grall
Hi Tamas, On 30/12/2019 18:15, Tamas K Lengyel wrote: On Mon, Dec 30, 2019 at 10:59 AM Roger Pau Monné wrote: On Thu, Dec 19, 2019 at 08:58:01AM -0700, Tamas K Lengyel wrote: On Thu, Dec 19, 2019 at 2:48 AM Roger Pau Monné wrote: On Wed, Dec 18, 2019 at 11:40:37AM -0800, Tamas K Lengyel

Re: [Xen-devel] [PATCH v2 00/20] VM forking

2019-12-30 Thread Tamas K Lengyel
On Mon, Dec 30, 2019 at 10:59 AM Roger Pau Monné wrote: > > On Thu, Dec 19, 2019 at 08:58:01AM -0700, Tamas K Lengyel wrote: > > On Thu, Dec 19, 2019 at 2:48 AM Roger Pau Monné > > wrote: > > > > > > On Wed, Dec 18, 2019 at 11:40:37AM -0800, Tamas K Lengyel wrote: > > > > The following series

Re: [Xen-devel] [PATCH v2 00/20] VM forking

2019-12-30 Thread Roger Pau Monné
On Thu, Dec 19, 2019 at 08:58:01AM -0700, Tamas K Lengyel wrote: > On Thu, Dec 19, 2019 at 2:48 AM Roger Pau Monné wrote: > > > > On Wed, Dec 18, 2019 at 11:40:37AM -0800, Tamas K Lengyel wrote: > > > The following series implements VM forking for Intel HVM guests to allow > > > for > > > the

[Xen-devel] Discovering L0 hypervisor features from L2

2019-12-30 Thread Wei Liu
Hi all One of the things required to make a Xen system run well on Hyper-V is to make Dom0 use as many paravirtualised interfaces as possible. For starters, we would really want Dom0 Linux kernel to use Hyper-V's PV drivers. But the core VMBus driver is gated by cpuid leaves. If cpuid returns

[Xen-devel] [qemu-mainline test] 145388: regressions - FAIL

2019-12-30 Thread osstest service owner
flight 145388 qemu-mainline real [real] http://logs.test-lab.xenproject.org/osstest/logs/145388/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-i386-freebsd10-i386 14 guest-saverestore fail REGR. vs. 144861

[Xen-devel] Proxying Hyper-V hypercalls from L2 to L0

2019-12-30 Thread Wei Liu
Hi all As much as I try to avoid writing code to proxy Hyper-V hypercalls, it seems unavoidable for PV guests, because Hyper-V requires hypercalls to be issued with CPL=0. This means for PV Dom0 I will need to add code in Xen to support Hyper-V's ABIs, along with appropriate validations. How

[Xen-devel] [PATCH v3 18/18] xen/tools: VM forking toolstack side

2019-12-30 Thread Tamas K Lengyel
Add necessary bits to implement "xl fork-vm", "xl fork-launch-dm" and "xl fork-reset" commands. The process is split in two to allow tools needing access to the new VM as fast as possible after it was forked. It is expected that under certain use-cases the second command that launches QEMU will be

[Xen-devel] [PATCH v3 17/18] x86/mem_sharing: reset a fork

2019-12-30 Thread Tamas K Lengyel
Implement hypercall that allows a fork to shed all memory that got allocated for it during its execution and re-load its vCPU context from the parent VM. This allows the forked VM to reset into the same state the parent VM is in a faster way then creating a new fork would be. Measurements show

[Xen-devel] [PATCH v3 13/18] x86/mem_sharing: Skip xen heap pages in memshr nominate

2019-12-30 Thread Tamas K Lengyel
Trying to share these would fail anyway, better to skip them early. Signed-off-by: Tamas K Lengyel --- xen/arch/x86/mm/mem_sharing.c | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/xen/arch/x86/mm/mem_sharing.c b/xen/arch/x86/mm/mem_sharing.c index

[Xen-devel] [PATCH v3 06/18] x86/mem_sharing: define mem_sharing_domain to hold some scattered variables

2019-12-30 Thread Tamas K Lengyel
Create struct mem_sharing_domain under hvm_domain and move mem sharing variables into it from p2m_domain and hvm_domain. Expose the mem_sharing_enabled macro to be used consistently across Xen. Remove some duplicate calls to mem_sharing_enabled in mem_sharing.c Signed-off-by: Tamas K Lengyel

[Xen-devel] [PATCH v3 07/18] x86/mem_sharing: Use INVALID_MFN and p2m_is_shared in relinquish_shared_pages

2019-12-30 Thread Tamas K Lengyel
While using _mfn(0) is of no consequence during teardown, INVALID_MFN is the correct value that should be used. Signed-off-by: Tamas K Lengyel --- xen/arch/x86/mm/mem_sharing.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/xen/arch/x86/mm/mem_sharing.c

[Xen-devel] [PATCH v3 15/18] xen/mem_sharing: VM forking

2019-12-30 Thread Tamas K Lengyel
VM forking is the process of creating a domain with an empty memory space and a parent domain specified from which to populate the memory when necessary. For the new domain to be functional the VM state is copied over as part of the fork operation (HVM params, hap allocation, etc). Signed-off-by:

[Xen-devel] [PATCH v3 08/18] x86/mem_sharing: Make add_to_physmap static and shorten name

2019-12-30 Thread Tamas K Lengyel
It's not being called from outside mem_sharing.c Signed-off-by: Tamas K Lengyel --- xen/arch/x86/mm/mem_sharing.c | 7 --- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/xen/arch/x86/mm/mem_sharing.c b/xen/arch/x86/mm/mem_sharing.c index 95e75ff298..84b9f130b9 100644 ---

[Xen-devel] [PATCH v3 10/18] x86/mem_sharing: Replace MEM_SHARING_DEBUG with gdprintk

2019-12-30 Thread Tamas K Lengyel
Using XENLOG_ERR level since this is only used in debug paths (ie. it's expected the user already has loglvl=all set). Signed-off-by: Tamas K Lengyel --- xen/arch/x86/mm/mem_sharing.c | 86 +-- 1 file changed, 43 insertions(+), 43 deletions(-) diff --git

[Xen-devel] [PATCH v3 14/18] x86/mem_sharing: check page type count earlier

2019-12-30 Thread Tamas K Lengyel
Signed-off-by: Tamas K Lengyel --- xen/arch/x86/mm/mem_sharing.c | 13 ++--- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/xen/arch/x86/mm/mem_sharing.c b/xen/arch/x86/mm/mem_sharing.c index baa3e35ded..ecbe40545d 100644 --- a/xen/arch/x86/mm/mem_sharing.c +++

[Xen-devel] [PATCH v3 09/18] x86/mem_sharing: Convert MEM_SHARING_DESTROY_GFN to a bool

2019-12-30 Thread Tamas K Lengyel
MEM_SHARING_DESTROY_GFN is used on the 'flags' bitfield during unsharing. However, the bitfield is not used for anything else, so just convert it to a bool instead. Signed-off-by: Tamas K Lengyel --- xen/arch/x86/mm/mem_sharing.c | 9 - xen/include/asm-x86/mem_sharing.h | 5 ++--- 2

[Xen-devel] [PATCH v3 16/18] xen/mem_access: Use __get_gfn_type_access in set_mem_access

2019-12-30 Thread Tamas K Lengyel
Use __get_gfn_type_access instead of p2m->get_entry to trigger page-forking when the mem_access permission is being set on a page that has not yet been copied over from the parent. Signed-off-by: Tamas K Lengyel --- xen/arch/x86/mm/mem_access.c | 5 ++--- 1 file changed, 2 insertions(+), 3

[Xen-devel] [PATCH v3 00/18] VM forking

2019-12-30 Thread Tamas K Lengyel
The following series implements VM forking for Intel HVM guests to allow for the fast creation of identical VMs without the assosciated high startup costs of booting or restoring the VM from a savefile. JIRA issue: https://xenproject.atlassian.net/browse/XEN-89 The main design goal with this

[Xen-devel] [PATCH v3 02/18] xen/x86: Make hap_get_allocation accessible

2019-12-30 Thread Tamas K Lengyel
During VM forking we'll copy the parent domain's parameters to the client, including the HAP shadow memory setting that is used for storing the domain's EPT. We'll copy this in the hypervisor instead doing it during toolstack launch to allow the domain to start executing and unsharing memory

[Xen-devel] [PATCH v3 12/18] x86/mem_sharing: Enable mem_sharing on first memop

2019-12-30 Thread Tamas K Lengyel
It is wasteful to require separate hypercalls to enable sharing on both the parent and the client domain during VM forking. To speed things up we enable sharing on the first memop in case it wasn't already enabled. Signed-off-by: Tamas K Lengyel --- xen/arch/x86/mm/mem_sharing.c | 36

[Xen-devel] [PATCH v3 01/18] x86/hvm: introduce hvm_copy_context_and_params

2019-12-30 Thread Tamas K Lengyel
Currently the hvm parameters are only accessible via the HVMOP hypercalls. In this patch we introduce a new function that can copy both the hvm context and parameters directly into a target domain. We'll use this function during VM forking operation. There are no functional changes to

[Xen-devel] [PATCH v3 03/18] x86/mem_sharing: make get_two_gfns take locks conditionally

2019-12-30 Thread Tamas K Lengyel
During VM forking the client lock will already be taken. Signed-off-by: Tamas K Lengyel Acked-by: Andrew Coopers --- xen/arch/x86/mm/mem_sharing.c | 11 ++- xen/include/asm-x86/p2m.h | 10 +- 2 files changed, 11 insertions(+), 10 deletions(-) diff --git

[Xen-devel] [PATCH v3 04/18] x86/mem_sharing: drop flags from mem_sharing_unshare_page

2019-12-30 Thread Tamas K Lengyel
All callers pass 0 in. Signed-off-by: Tamas K Lengyel Reviewed-by: Wei Liu --- xen/arch/x86/hvm/hvm.c| 2 +- xen/arch/x86/mm/p2m.c | 5 ++--- xen/common/memory.c | 2 +- xen/include/asm-x86/mem_sharing.h | 8 +++- 4 files changed, 7 insertions(+), 10

[Xen-devel] [PATCH v3 11/18] x86/mem_sharing: ASSERT that p2m_set_entry succeeds

2019-12-30 Thread Tamas K Lengyel
Signed-off-by: Tamas K Lengyel --- xen/arch/x86/mm/mem_sharing.c | 42 +-- 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/xen/arch/x86/mm/mem_sharing.c b/xen/arch/x86/mm/mem_sharing.c index 93e7605900..3f36cd6bbc 100644 ---

[Xen-devel] [PATCH v3 05/18] x86/mem_sharing: don't try to unshare twice during page fault

2019-12-30 Thread Tamas K Lengyel
The page was already tried to be unshared in get_gfn_type_access. If that didn't work, then trying again is pointless. Don't try to send vm_event again either, simply check if there is a ring or not. Signed-off-by: Tamas K Lengyel --- v3: use gprintk when the domain is going to crash ---

Re: [Xen-devel] [PATCH] tools/libxl: Reposition build_pre() logic between architectures

2019-12-30 Thread Julien Grall
Hi, On 30/12/2019 13:38, Andrew Cooper wrote: On 30/12/2019 13:15, Julien Grall wrote: Hi Andrew, On 27/12/2019 13:45, Andrew Cooper wrote: The call to xc_domain_disable_migrate() is made only from x86, while its handling in Xen is common.  Move it to the libxl__build_pre().

Re: [Xen-devel] [PATCH 4/8] x86/hyperv: setup hypercall page

2019-12-30 Thread Andrew Cooper
On 30/12/2019 13:33, Wei Liu wrote: > On Mon, Dec 30, 2019 at 12:55:22PM +, Andrew Cooper wrote: >> On 29/12/2019 18:33, Wei Liu wrote: >>> @@ -71,6 +72,40 @@ const struct hypervisor_ops *__init hyperv_probe(void) >>> return >>> } >>> >>> +static void __init setup_hypercall_page(void)

Re: [Xen-devel] [PATCH] tools/libxl: Reposition build_pre() logic between architectures

2019-12-30 Thread Andrew Cooper
On 30/12/2019 13:15, Julien Grall wrote: > Hi Andrew, > > On 27/12/2019 13:45, Andrew Cooper wrote: >> The call to xc_domain_disable_migrate() is made only from x86, while its >> handling in Xen is common.  Move it to the libxl__build_pre(). >> >> hvm_set_conf_params(),

Re: [Xen-devel] [PATCH 5/8] x86/hyperv: provide Hyper-V hypercall functions

2019-12-30 Thread Wei Liu
On Mon, Dec 30, 2019 at 01:04:33PM +, Andrew Cooper wrote: > On 29/12/2019 18:33, Wei Liu wrote: > > These functions will be used later to make hypercalls to Hyper-V. > > > > I couldn't find reference in TLFS that Hyper-V clobbers flags and > > r9-r11, but Linux's commit message says it does.

Re: [Xen-devel] [PATCH 4/8] x86/hyperv: setup hypercall page

2019-12-30 Thread Wei Liu
On Mon, Dec 30, 2019 at 12:55:22PM +, Andrew Cooper wrote: > On 29/12/2019 18:33, Wei Liu wrote: > > @@ -71,6 +72,40 @@ const struct hypervisor_ops *__init hyperv_probe(void) > > return > > } > > > > +static void __init setup_hypercall_page(void) > > +{ > > +union

Re: [Xen-devel] [PATCH] tools/libxl: Reposition build_pre() logic between architectures

2019-12-30 Thread Julien Grall
Hi Andrew, On 27/12/2019 13:45, Andrew Cooper wrote: The call to xc_domain_disable_migrate() is made only from x86, while its handling in Xen is common. Move it to the libxl__build_pre(). hvm_set_conf_params(), hvm_set_viridian_features(), hvm_set_mca_capabilities(), and the altp2m logic is

Re: [Xen-devel] [PATCH 5/8] x86/hyperv: provide Hyper-V hypercall functions

2019-12-30 Thread Andrew Cooper
On 29/12/2019 18:33, Wei Liu wrote: > These functions will be used later to make hypercalls to Hyper-V. > > I couldn't find reference in TLFS that Hyper-V clobbers flags and > r9-r11, but Linux's commit message says it does. Err on the safe side. > > Signed-off-by: Wei Liu > --- >

Re: [Xen-devel] [PATCH] tools/libxl: Reposition build_pre() logic between architectures

2019-12-30 Thread Wei Liu
On Fri, Dec 27, 2019 at 02:50:58PM +0100, Jan Beulich wrote: > On 27.12.2019 14:45, Andrew Cooper wrote: > > The call to xc_domain_disable_migrate() is made only from x86, while its > > handling in Xen is common. Move it to the libxl__build_pre(). > > > > hvm_set_conf_params(),

Re: [Xen-devel] [PATCH 4/8] x86/hyperv: setup hypercall page

2019-12-30 Thread Andrew Cooper
On 29/12/2019 18:33, Wei Liu wrote: > @@ -71,6 +72,40 @@ const struct hypervisor_ops *__init hyperv_probe(void) > return > } > > +static void __init setup_hypercall_page(void) > +{ > +union hv_x64_msr_hypercall_contents hypercall_msr; > + > +/* Unfortunately there isn't a really

Re: [Xen-devel] [PATCH] tools/libxl: Reposition build_pre() logic between architectures

2019-12-30 Thread Wei Liu
On Fri, Dec 27, 2019 at 01:45:16PM +, Andrew Cooper wrote: > The call to xc_domain_disable_migrate() is made only from x86, while its > handling in Xen is common. Move it to the libxl__build_pre(). > > hvm_set_conf_params(), hvm_set_viridian_features(), > hvm_set_mca_capabilities(), and the

[Xen-devel] [xen-unstable test] 145377: regressions - FAIL

2019-12-30 Thread osstest service owner
flight 145377 xen-unstable real [real] http://logs.test-lab.xenproject.org/osstest/logs/145377/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-amd64-qemuu-nested-intel 17 debian-hvm-install/l1/l2 fail REGR. vs. 145025 Tests