[Xen-devel] [xen-unstable test] 145202: regressions - FAIL

2019-12-24 Thread osstest service owner
flight 145202 xen-unstable real [real] http://logs.test-lab.xenproject.org/osstest/logs/145202/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-amd64-qemuu-nested-intel 17 debian-hvm-install/l1/l2 fail REGR. vs. 145025

[Xen-devel] [qemu-mainline test] 145195: regressions - FAIL

2019-12-24 Thread osstest service owner
flight 145195 qemu-mainline real [real] http://logs.test-lab.xenproject.org/osstest/logs/145195/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-i386-freebsd10-amd64 14 guest-saverestore fail REGR. vs. 144861

[Xen-devel] [xen-unstable test] 145185: regressions - FAIL

2019-12-24 Thread osstest service owner
flight 145185 xen-unstable real [real] http://logs.test-lab.xenproject.org/osstest/logs/145185/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-amd64-qemuu-nested-intel 17 debian-hvm-install/l1/l2 fail REGR. vs. 145025 Tests

Re: [Xen-devel] [PATCH v4 2/6] golang/xenlight: begin Go to C type marshaling

2019-12-24 Thread Nick Rosbrook
> I realize this is all generated code, but there's still a massive amount > of duplication here, which will at very least cause code bloat. I think > it should be possible to do this all at once with a defer, like this: > > func (x *SpiceInfo) toC() (xc C.libxl_spice_info, err error) { >

Re: [Xen-devel] [PATCH] x86/hvm: always expose x2APIC feature in max HVM cpuid policy

2019-12-24 Thread Roger Pau Monné
On Tue, Dec 24, 2019 at 04:00:27PM +, Andrew Cooper wrote: > On 24/12/2019 12:42, Roger Pau Monné wrote: > > On Tue, Dec 24, 2019 at 12:23:12PM +, Andrew Cooper wrote: > >> On 24/12/2019 10:18, Roger Pau Monne wrote: > >>> On hardware without x2APIC support Xen emulated local APIC will >

[Xen-devel] [xen-unstable-smoke test] 145194: tolerable all pass - PUSHED

2019-12-24 Thread osstest service owner
flight 145194 xen-unstable-smoke real [real] http://logs.test-lab.xenproject.org/osstest/logs/145194/ Failures :-/ but no regressions. Tests which did not succeed, but are not blocking: test-amd64-amd64-libvirt 13 migrate-support-checkfail never pass test-arm64-arm64-xl-xsm

Re: [Xen-devel] [PATCH] x86/hvm: always expose x2APIC feature in max HVM cpuid policy

2019-12-24 Thread Andrew Cooper
On 24/12/2019 12:42, Roger Pau Monné wrote: > On Tue, Dec 24, 2019 at 12:23:12PM +, Andrew Cooper wrote: >> On 24/12/2019 10:18, Roger Pau Monne wrote: >>> On hardware without x2APIC support Xen emulated local APIC will >>> provide such mode, and hence the feature should be set in the maximum

[Xen-devel] [PATCH 10/12] docs/migration: Specify X86_{CPUID, MSR}_POLICY records

2019-12-24 Thread Andrew Cooper
These two records move blobs from the XEN_DOMCTL_{get,set}_cpu_policy hypercall. Signed-off-by: Andrew Cooper --- CC: George Dunlap CC: Ian Jackson CC: Jan Beulich CC: Konrad Rzeszutek Wilk CC: Stefano Stabellini CC: Tim Deegan CC: Wei Liu CC: Julien Grall CC: Marek Marczykowski-Górecki

[Xen-devel] [PATCH 12/12] libxc/save: Write X86_{CPUID, MSR}_DATA records

2019-12-24 Thread Andrew Cooper
With all other plumbing in place, obtain the CPU Policy from Xen and write it into the migration stream. Signed-off-by: Andrew Cooper --- CC: Ian Jackson CC: Wei Liu --- tools/libxc/xc_sr_common_x86.c | 50 tools/libxc/xc_sr_common_x86.h | 6 +

[Xen-devel] [PATCH 11/12] libxc/restore: Handle X86_{CPUID, MSR}_DATA records

2019-12-24 Thread Andrew Cooper
For now, the data are just stashed, and discarded at the end. This will be addressed when the TODO in x86_static_data_complete() is addressed. No practical change to behaviour - this is all plumbing work. Signed-off-by: Andrew Cooper --- CC: Ian Jackson CC: Wei Liu ---

Re: [Xen-devel] [PATCH] x86/vvmx: virtualize x2APIC mode and APIC accesses can't both be enabled

2019-12-24 Thread Andrew Cooper
On 24/12/2019 15:32, Roger Pau Monne wrote: > According to the Intel SDM, "virtualize x2APIC mode" and "virtualize > APIC accesses" can't be enabled at the same time, or else a > vm{launch/entry} failure will happen. This was seen when running Xen > nested and with x2APIC enabled: > > Dec 23

[Xen-devel] [PATCH] x86/vvmx: virtualize x2APIC mode and APIC accesses can't both be enabled

2019-12-24 Thread Roger Pau Monne
According to the Intel SDM, "virtualize x2APIC mode" and "virtualize APIC accesses" can't be enabled at the same time, or else a vm{launch/entry} failure will happen. This was seen when running Xen nested and with x2APIC enabled: Dec 23 20:06:54.444025 (XEN) d3v0 VMLAUNCH error: 0x7 [...] Dec 23

[Xen-devel] [PATCH 01/12] libxc/save: Shrink code volume where possible

2019-12-24 Thread Andrew Cooper
A property of how the error handling (0 on success, nonzero otherwise) allows these calls to be chained together with the ternary operatior. No functional change, but far less boilerplate code. Signed-off-by: Andrew Cooper --- CC: Ian Jackson CC: Wei Liu --- tools/libxc/xc_sr_save.c

[Xen-devel] [PATCH 04/12] libxc/migration: Adjust layout of struct xc_sr_context

2019-12-24 Thread Andrew Cooper
We are shortly going to want to introduce some common x86 fields, so having x86_pv and x86_hvm as the top level objects is a problem. Insert a surrounding struct x86 and drop the x86 prefix from the pv/hvm objects. No functional change. Signed-off-by: Andrew Cooper --- CC: Ian Jackson CC: Wei

[Xen-devel] [PATCH 05/12] tools/migration: Drop IHDR_VERSION constant from libxc and python

2019-12-24 Thread Andrew Cooper
Migration v3 is in the process of being introduced, meaning that the code has to cope with both versions. Use an explicit 2 for now. For the verify-stream-v2 and convert-legacy-stream scripts, update text to say "v2 (or later)". What matters is the distinction vs legacy streams. Signed-off-by:

[Xen-devel] [PATCH 02/12] libxc/restore: Introduce functionality to simplify blob handling

2019-12-24 Thread Andrew Cooper
During migration, we buffer several blobs of data which ultimately need handing back to Xen at an appropriate time. Currently, this is all handled in an ad-hoc manner, but more blobs are soon going to be added. Introduce xc_sr_blob to encapsulate a ptr/size pair, and update_blob() to handle the

[Xen-devel] [PATCH 03/12] libxc/migration: Rationalise the 'checkpointed' field to 'stream_type'

2019-12-24 Thread Andrew Cooper
Originally, 'checkpointed' was a boolean signalling the difference between a plain and a Remus stream. COLO was added later, but several bits of code retained boolean-style logic. While correct, it is confusing to follow. Additionally, XC_MIG_STREAM_NONE means "no checkpoints" but reads as "no

[Xen-devel] [PATCH 06/12] docs/migration Specify migration v3 and STATIC_DATA_END

2019-12-24 Thread Andrew Cooper
Migration data can be split into two parts - that which is invariant of guest execution, and that which is not. Separate these two with the STATIC_DATA_END record. The short term, we want to move the x86 CPU Policy data into the stream. In the longer term, we want to provisionally send the

[Xen-devel] [PATCH 07/12] python/migration: Update validation logic to understand a v3 stream

2019-12-24 Thread Andrew Cooper
Signed-off-by: Andrew Cooper --- CC: Marek Marczykowski-Górecki --- tools/python/xen/migration/libxc.py | 26 ++ 1 file changed, 22 insertions(+), 4 deletions(-) diff --git a/tools/python/xen/migration/libxc.py b/tools/python/xen/migration/libxc.py index

[Xen-devel] [PATCH 09/12] libxc/save: Write a v3 stream

2019-12-24 Thread Andrew Cooper
Introduce a new static_data() hook which is responsible for writing out any static data records. The HVM side continues to be a no-op, while the PV side moves write_x86_pv_info() into this earlier hook. The the common code writes out a STATIC_DATA_END record, and the stream version is bumped to

[Xen-devel] [PATCH 08/12] libxc/restore: Support v3 streams, and cope with v2 compatibilty

2019-12-24 Thread Andrew Cooper
Introduce a static_data_complete() hook which is called when a STATIC_DATA_END record is found (v3) or inferred (v2). Modify handle_page_data() and handle_x86_pv_p2m_frames() to infer the position in v2 streams. The implementation of x86_static_data_complete() needs to wait until more plumbing

[Xen-devel] [PATCH 00/12] Support CPUID/MSR data in migration streams

2019-12-24 Thread Andrew Cooper
As an early christmas present, here is the pumbing work to include CPUID/MSR data in migration streams. For bisectability reasons, this is a large amount of plumbing with no overall change in behaviour. Switching to use the CPUID/MSR data in the stream is going to require some careful

[Xen-devel] [qemu-mainline test] 145176: regressions - FAIL

2019-12-24 Thread osstest service owner
flight 145176 qemu-mainline real [real] http://logs.test-lab.xenproject.org/osstest/logs/145176/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-i386-freebsd10-amd64 14 guest-saverestore fail REGR. vs. 144861

Re: [Xen-devel] [PATCH v4 2/6] golang/xenlight: begin Go to C type marshaling

2019-12-24 Thread George Dunlap
On 12/23/19 3:17 PM, Nick Rosbrook wrote: > @@ -92,6 +131,43 @@ func (x *SpiceInfo) fromC(xc *C.libxl_spice_info) error { > return nil > } > > +func (x *SpiceInfo) toC() (xc C.libxl_spice_info, err error) { > + C.libxl_spice_info_init() > + xc.enable, err = x.Enable.toC() > +

Re: [Xen-devel] [xen-unstable bisection] complete test-amd64-amd64-qemuu-nested-intel

2019-12-24 Thread Roger Pau Monné
On Sun, Dec 22, 2019 at 08:52:02PM +, osstest service owner wrote: > branch xen-unstable > xenbranch xen-unstable > job test-amd64-amd64-qemuu-nested-intel > testid debian-hvm-install/l1/l2 > > Tree: linux git://xenbits.xen.org/linux-pvops.git > Tree: linuxfirmware

[Xen-devel] [PATCH 1/2] x86/hvm: improve performance of HVMOP_flush_tlbs

2019-12-24 Thread Roger Pau Monne
There's no need to call paging_update_cr3 unless CR3 trapping is enabled, and that's only the case when using shadow paging or when requested for introspection purposes, otherwise there's no need to pause all the vCPUs of the domain in order to perform the flush. Check whether CR3 trapping is

[Xen-devel] [PATCH 0/2] x86: improve assisted tlb flush and use it in guest mode

2019-12-24 Thread Roger Pau Monne
Hello, The following series aims to improve the tlb flush times when running nested Xen, and it's specially beneficial when running in shim mode. See patch #2 for a comparison on the performance of the L0 assisted flush vs using x2APIC shorthand. Thanks, Roger. Roger Pau Monne (2): x86/hvm:

[Xen-devel] [PATCH 2/2] x86/tlb: use Xen L0 assisted TLB flush when available

2019-12-24 Thread Roger Pau Monne
Use Xen's L0 HVMOP_flush_tlbs hypercall when available in order to perform flushes. This greatly increases the performance of tlb flushes when running with a high amount of vCPUs as a Xen guest, and is specially important when running in shim mode. The following figures are from a PV guest

[Xen-devel] [PATCH 1/6] libxl: add definition of INVALID_DOMID to the API

2019-12-24 Thread Paul Durrant
Currently both xl and libxl have internal definitions of INVALID_DOMID which happen to be identical. However, for the purposes of describing the behaviour of libxl_domain_create_new/restore() it is useful to have a specified invalid value for a domain id. This patch therefore moves the libxl

[Xen-devel] [PATCH 0/6] xl/libxl: allow creation of domains with a specified domid

2019-12-24 Thread Paul Durrant
Paul Durrant (6): libxl: add definition of INVALID_DOMID to the API libxl_create: make 'soft reset' explicit domctl: return EEXIST from XEN_DOMCTL_createdomain... domctl: set XEN_DOMCTL_createdomain 'rover' if valid domid is specified libxl: allow creation of domains with specified

[Xen-devel] [PATCH 6/6] xl: allow specified domain id to be used for create, restore and migrate

2019-12-24 Thread Paul Durrant
This patch adds the option to use a specified domain id to be used for the create, restore and migrate lifecycle operations and documentation thereof. The specified id may be numeric or, in all cases, one of two special values. The value 'random' will cause libxl to use a randomly chosen domain

[Xen-devel] [PATCH 4/6] domctl: set XEN_DOMCTL_createdomain 'rover' if valid domid is specified

2019-12-24 Thread Paul Durrant
The value of 'rover' is the value at which Xen will start searching for an unused domid if none is specified. Currently it is only updated when a domid is automatically chosen, rather than specified by the caller, which makes it very hard to describe Xen's rationale in choosing domids in an

[Xen-devel] [PATCH 5/6] libxl: allow creation of domains with specified or random domid

2019-12-24 Thread Paul Durrant
This patch modifies do_domain_create() to use the value of domid that is passed in. A new 'special value' - RANDOM_DOMID - is added into the API and this, INVALID_DOMID or any valid domid is passed unmodified to libxl__domain_make(). Any other invalid domid value will cause an error. If

[Xen-devel] [PATCH 2/6] libxl_create: make 'soft reset' explicit

2019-12-24 Thread Paul Durrant
The 'soft reset' code path in libxl__domain_make() is currently taken if a valid domid is passed into the function. A subsequent patch will enable higher levels of the toolstack to determine the domid of newly created or restored domains and therefore this criteria for choosing 'soft reset' will

[Xen-devel] [PATCH 3/6] domctl: return EEXIST from XEN_DOMCTL_createdomain...

2019-12-24 Thread Paul Durrant
...if a specified domid is already in use. XEN_DOMCTL_createdomain allows a domid to be specified by its caller and will correctly fail if that domid is already in use. However the errno returned in this case will be EINVAL, making it indistinguishable from several other failures. Also a value of

[Xen-devel] [xen-unstable-smoke test] 145182: tolerable all pass - PUSHED

2019-12-24 Thread osstest service owner
flight 145182 xen-unstable-smoke real [real] http://logs.test-lab.xenproject.org/osstest/logs/145182/ Failures :-/ but no regressions. Tests which did not succeed, but are not blocking: test-amd64-amd64-libvirt 13 migrate-support-checkfail never pass test-arm64-arm64-xl-xsm

[Xen-devel] [PATCH] x86/flush: use APIC ALLBUT destination shorthand when possible

2019-12-24 Thread Roger Pau Monne
If the flush mask matches the mask of online CPUs use the APIC ALLBUT destination shorthand in order to send an IPI to all CPUs on the system except the current one. This can only be safely used when no CPU hotplug or unplug operations are taking place, no offline CPUs or those have been onlined

Re: [Xen-devel] [PATCH] x86/hvm: always expose x2APIC feature in max HVM cpuid policy

2019-12-24 Thread Roger Pau Monné
On Tue, Dec 24, 2019 at 12:23:12PM +, Andrew Cooper wrote: > On 24/12/2019 10:18, Roger Pau Monne wrote: > > On hardware without x2APIC support Xen emulated local APIC will > > provide such mode, and hence the feature should be set in the maximum > > HVM cpuid policy. > > > > Not exposing it

Re: [Xen-devel] [PATCH] x86/hvm: always expose x2APIC feature in max HVM cpuid policy

2019-12-24 Thread Andrew Cooper
On 24/12/2019 10:18, Roger Pau Monne wrote: > On hardware without x2APIC support Xen emulated local APIC will > provide such mode, and hence the feature should be set in the maximum > HVM cpuid policy. > > Not exposing it in the maximum policy results in HVM domains not > getting such feature

[Xen-devel] [ovmf test] 145179: all pass - PUSHED

2019-12-24 Thread osstest service owner
flight 145179 ovmf real [real] http://logs.test-lab.xenproject.org/osstest/logs/145179/ Perfect :-) All tests in this flight passed as required version targeted for testing: ovmf a5abd9cc2cebe7fac001f7bb7b647c47cf54af1a baseline version: ovmf

[Xen-devel] [xen-unstable test] 145168: regressions - FAIL

2019-12-24 Thread osstest service owner
flight 145168 xen-unstable real [real] http://logs.test-lab.xenproject.org/osstest/logs/145168/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 7 xen-boot fail REGR. vs. 145025

[Xen-devel] [PATCH] x86/hvm: always expose x2APIC feature in max HVM cpuid policy

2019-12-24 Thread Roger Pau Monne
On hardware without x2APIC support Xen emulated local APIC will provide such mode, and hence the feature should be set in the maximum HVM cpuid policy. Not exposing it in the maximum policy results in HVM domains not getting such feature exposed unless it's also supported by the underlying

Re: [Xen-devel] [PATCH V6 4/4] x86/mm: Make use of the default access param from xc_altp2m_create_view

2019-12-24 Thread Alexandru Stefan ISAILA
On 24.12.2019 10:48, George Dunlap wrote: > On 12/23/19 2:04 PM, Alexandru Stefan ISAILA wrote: >> At this moment the default_access param from xc_altp2m_create_view is >> not used. >> >> This patch assigns default_access to p2m->default_access at the time of >> initializing a new altp2m view. >

Re: [Xen-devel] [PATCH V6 3/4] x86/mm: Pull out the p2m specifics from p2m_init_altp2m_ept

2019-12-24 Thread George Dunlap
On 12/24/19 10:08 AM, Alexandru Stefan ISAILA wrote: > > > On 24.12.2019 10:01, George Dunlap wrote: >> On 12/23/19 2:04 PM, Alexandru Stefan ISAILA wrote: >> >> Why? >> > > This was a request from Jan. Yes, I saw the Requested-by. It still needs an explanation. -George

Re: [Xen-devel] [PATCH V6 3/4] x86/mm: Pull out the p2m specifics from p2m_init_altp2m_ept

2019-12-24 Thread Alexandru Stefan ISAILA
On 24.12.2019 10:01, George Dunlap wrote: > On 12/23/19 2:04 PM, Alexandru Stefan ISAILA wrote: > > Why? > This was a request from Jan. Alex ___ Xen-devel mailing list Xen-devel@lists.xenproject.org

[Xen-devel] [libvirt test] 145173: regressions - FAIL

2019-12-24 Thread osstest service owner
flight 145173 libvirt real [real] http://logs.test-lab.xenproject.org/osstest/logs/145173/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-arm64-arm64-libvirt-qcow2 15 guest-start/debian.repeat fail REGR. vs. 145054 Tests which did

Re: [Xen-devel] [PATCH v4 1/6] golang/xenlight: implement array C to Go marshaling

2019-12-24 Thread George Dunlap
On 12/23/19 3:17 PM, Nick Rosbrook wrote: > Signed-off-by: Nick Rosbrook Reviewed-by: George Dunlap I'm going to check in this patch, as well as 5/6 and 6/6 right now. ___ Xen-devel mailing list Xen-devel@lists.xenproject.org

Re: [Xen-devel] [PATCH V6 2/4] x86/altp2m: Add hypercall to set a range of sve bits

2019-12-24 Thread George Dunlap
On 12/24/19 9:04 AM, Alexandru Stefan ISAILA wrote: > +/* > + * Set/clear the #VE suppress bit for multiple pages. Only available on > VMX. > + */ > +int p2m_set_suppress_ve_multi(struct domain *d, > + struct xen_hvm_altp2m_suppress_ve_multi

Re: [Xen-devel] [PATCH V6 2/4] x86/altp2m: Add hypercall to set a range of sve bits

2019-12-24 Thread Alexandru Stefan ISAILA
+/* + * Set/clear the #VE suppress bit for multiple pages. Only available on VMX. + */ +int p2m_set_suppress_ve_multi(struct domain *d, + struct xen_hvm_altp2m_suppress_ve_multi *sve) +{ +struct p2m_domain *host_p2m =

Re: [Xen-devel] [PATCH V6 2/4] x86/altp2m: Add hypercall to set a range of sve bits

2019-12-24 Thread George Dunlap
On 12/24/19 8:48 AM, Alexandru Stefan ISAILA wrote: > > > On 24.12.2019 10:30, George Dunlap wrote: >> On 12/23/19 2:04 PM, Alexandru Stefan ISAILA wrote: >>> By default the sve bits are not set. >>> This patch adds a new hypercall, xc_altp2m_set_supress_ve_multi(), >>> to set a range of sve

Re: [Xen-devel] [PATCH V6 2/4] x86/altp2m: Add hypercall to set a range of sve bits

2019-12-24 Thread Alexandru Stefan ISAILA
On 24.12.2019 10:30, George Dunlap wrote: > On 12/23/19 2:04 PM, Alexandru Stefan ISAILA wrote: >> By default the sve bits are not set. >> This patch adds a new hypercall, xc_altp2m_set_supress_ve_multi(), >> to set a range of sve bits. >> The core function, p2m_set_suppress_ve_multi(), does not

Re: [Xen-devel] [PATCH V6 4/4] x86/mm: Make use of the default access param from xc_altp2m_create_view

2019-12-24 Thread George Dunlap
On 12/23/19 2:04 PM, Alexandru Stefan ISAILA wrote: > At this moment the default_access param from xc_altp2m_create_view is > not used. > > This patch assigns default_access to p2m->default_access at the time of > initializing a new altp2m view. That's certainly not what it looks like. It looks

Re: [Xen-devel] [PATCH V6 2/4] x86/altp2m: Add hypercall to set a range of sve bits

2019-12-24 Thread George Dunlap
On 12/23/19 2:04 PM, Alexandru Stefan ISAILA wrote: > By default the sve bits are not set. > This patch adds a new hypercall, xc_altp2m_set_supress_ve_multi(), > to set a range of sve bits. > The core function, p2m_set_suppress_ve_multi(), does not brake in case *break > of a error and it is

[Xen-devel] [ovmf test] 145172: all pass - PUSHED

2019-12-24 Thread osstest service owner
flight 145172 ovmf real [real] http://logs.test-lab.xenproject.org/osstest/logs/145172/ Perfect :-) All tests in this flight passed as required version targeted for testing: ovmf 123b720eeb371e0a31eb727bcf59255b584e355f baseline version: ovmf

Re: [Xen-devel] [PATCH V6 3/4] x86/mm: Pull out the p2m specifics from p2m_init_altp2m_ept

2019-12-24 Thread George Dunlap
On 12/23/19 2:04 PM, Alexandru Stefan ISAILA wrote: Why? -George ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel