[Xen-devel] [PATCH 3/3] lz4: fix system halt at boot kernel on x86_64

2019-12-04 Thread Jan Beulich
From: Krzysztof Kolasa Sometimes, on x86_64, decompression fails with the following error: Decompressing Linux... Decoding failed -- System halted This condition is not needed for a 64bit kernel(from commit d5e7caf): if( ... || (op + COPYLENGTH) > oend) goto _output_error macro

[Xen-devel] [PATCH 1/3] lz4: refine commit 9143a6c55ef7 for the 64-bit case

2019-12-04 Thread Jan Beulich
I clearly went too far there: While the LZ4_WILDCOPY() instances indeed need prior guarding, LZ4_SECURECOPY() needs this only in the 32-bit case (where it simply aliases LZ4_WILDCOPY()). "cpy" can validly point (slightly) below "op" in these cases, due to cpy = op + length -

[Xen-devel] [PATCH 2/3] lz4: pull out constant tables

2019-12-04 Thread Jan Beulich
From: Rasmus Villemoes There's no reason to allocate the dec{32,64}table on the stack; it just wastes a bunch of instructions setting them up and, of course, also consumes quite a bit of stack. Using size_t for such small integers is a little excessive. $ scripts/bloat-o-meter /tmp/built-in.o

[Xen-devel] [PATCH 0/3] lz4: misc fixes / adjustments

2019-12-04 Thread Jan Beulich
1: refine commit 9143a6c55ef7 for the 64-bit case 2: pull out constant tables 3: fix system halt at boot kernel on x86_64 Only patch 1 is strictly meant to be considered for 4.13, albeit patch 3 fixes a similar problem (but none which would have been reported so far). Patches 2 and 3 are ports of

Re: [Xen-devel] [PATCH v4 2/9] x86: introduce a new set of APIs to manage Xen page tables

2019-12-04 Thread Jan Beulich
On 04.12.2019 18:54, Xia, Hongyan wrote: >> There's no need for the map/unmap functions to have a _new >> suffix, is there? > > I thought this was weird at first also, but what I find really useful > is that we can just change all call sites to the new API in small steps > without breaking.

[Xen-devel] [libvirt test] 144526: regressions - FAIL

2019-12-04 Thread osstest service owner
flight 144526 libvirt real [real] http://logs.test-lab.xenproject.org/osstest/logs/144526/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: build-amd64-libvirt 6 libvirt-buildfail REGR. vs. 144517 build-i386-libvirt

[Xen-devel] [xen-unstable test] 144522: tolerable FAIL - PUSHED

2019-12-04 Thread osstest service owner
flight 144522 xen-unstable real [real] http://logs.test-lab.xenproject.org/osstest/logs/144522/ Failures :-/ but no regressions. Regressions which are regarded as allowable (not blocking): test-amd64-amd64-xl-rtds 16 guest-localmigrate fail REGR. vs. 144512 Tests which did not

[Xen-devel] [ovmf test] 144524: all pass - PUSHED

2019-12-04 Thread osstest service owner
flight 144524 ovmf real [real] http://logs.test-lab.xenproject.org/osstest/logs/144524/ Perfect :-) All tests in this flight passed as required version targeted for testing: ovmf c9416efeef0d4a0554db01f3fd1cdaede14856d7 baseline version: ovmf

Re: [Xen-devel] bug: unable to LZ4 decompress ub1910 installer kernel when launching domU

2019-12-04 Thread Pry Mar
On 12/4/19, Jan Beulich wrote: > On 04.12.2019 08:14, Jeremi Piotrowski wrote: >> Any suggestions how to proceed? > > Actually here's a better version (I think). > > Jan > > lz4: refine commit 9143a6c55ef7 for the 64-bit case > > I clearly went too far there: While the LZ4_WILDCOPY() instances

[Xen-devel] [PATCH v4 1/6] arm/arm64/xen: hypercall.h add includes guards

2019-12-04 Thread Pavel Tatashin
The arm and arm64 versions of hypercall.h are missing the include guards. This is needed because C inlines for privcmd_call are going to be added to the files. Also fix a comment. Signed-off-by: Pavel Tatashin --- arch/arm/include/asm/assembler.h | 2 +-

[Xen-devel] [PATCH v4 0/6] Use C inlines for uaccess

2019-12-04 Thread Pavel Tatashin
Changelog: v4: - Split the first patch into two as Julien Grall - Also as Mark Rutland suggested removed alternatives from __asm_flush_cache_user_range. v3: - Added Acked-by from Stefano Stabellini - Addressed comments from Mark Rutland v2: -

[Xen-devel] [PATCH v4 3/6] arm64: remove uaccess_ttbr0 asm macros from cache functions

2019-12-04 Thread Pavel Tatashin
We currently duplicate the logic to enable/disable uaccess via TTBR0, with C functions and assembly macros. This is a maintenenace burden and is liable to lead to subtle bugs, so let's get rid of the assembly macros, and always use the C functions. This requires refactoring some assembly functions

[Xen-devel] [PATCH v4 4/6] arm64: remove __asm_flush_icache_range

2019-12-04 Thread Pavel Tatashin
__asm_flush_icache_range is an alias to __asm_flush_cache_user_range, but now that these functions are called from C wrappers the fall through can instead be done at a higher level. Remove the __asm_flush_icache_range alias in assembly, and instead call __flush_cache_user_range() from

[Xen-devel] [PATCH v4 6/6] arm64: remove the rest of asm-uaccess.h

2019-12-04 Thread Pavel Tatashin
The __uaccess_ttbr0_disable and __uaccess_ttbr0_enable, are the last two macros defined in asm-uaccess.h. For now move them to entry.S where they are used. Eventually, these macros should be replaced with C wrappers to reduce the maintenance burden. Also, once these macros are unified with the C

[Xen-devel] [PATCH v4 2/6] arm/arm64/xen: use C inlines for privcmd_call

2019-12-04 Thread Pavel Tatashin
privcmd_call requires to enable access to userspace for the duration of the hypercall. Currently, this is done via assembly macros. Change it to C inlines instead. Signed-off-by: Pavel Tatashin Acked-by: Stefano Stabellini --- arch/arm/include/asm/xen/hypercall.h | 6 ++

[Xen-devel] [PATCH v4 5/6] arm64: move ARM64_HAS_CACHE_DIC/_IDC from asm to C

2019-12-04 Thread Pavel Tatashin
The assmbly functions __asm_flush_cache_user_range and __asm_invalidate_icache_range have alternatives: alternative_if ARM64_HAS_CACHE_DIC ... alternative_if ARM64_HAS_CACHE_IDC ... But, the implementation of those alternatives is trivial and therefore can be done in the C inline wrappers.

[Xen-devel] [xen-unstable-smoke test] 144523: tolerable all pass - PUSHED

2019-12-04 Thread osstest service owner
flight 144523 xen-unstable-smoke real [real] http://logs.test-lab.xenproject.org/osstest/logs/144523/ Failures :-/ but no regressions. Tests which did not succeed, but are not blocking: test-amd64-amd64-libvirt 13 migrate-support-checkfail never pass test-arm64-arm64-xl-xsm

Re: [Xen-devel] [PATCH] passthrough: add missed pcidevs_unlock following c/s cd7dedad820

2019-12-04 Thread Sander Eikelenboom
On 04/12/2019 22:31, Igor Druzhinin wrote: > The locking responsibilities have changed and a premature break in > this section now causes the following assertion: > > Assertion '!preempt_count()' failed at preempt.c:36 > > Reported-by: Sander Eikelenboom > Signed-off-by: Igor Druzhinin > --- >

[Xen-devel] [PATCH] passthrough: add missed pcidevs_unlock following c/s cd7dedad820

2019-12-04 Thread Igor Druzhinin
The locking responsibilities have changed and a premature break in this section now causes the following assertion: Assertion '!preempt_count()' failed at preempt.c:36 Reported-by: Sander Eikelenboom Signed-off-by: Igor Druzhinin --- xen/drivers/passthrough/pci.c | 1 + 1 file changed, 1

Re: [Xen-devel] xen-unstable (4.14 to be): Assertion '!preempt_count()' failed at preempt.c:36

2019-12-04 Thread Sander Eikelenboom
On 04/12/2019 18:30, Jan Beulich wrote: > On 04.12.2019 18:21, Sander Eikelenboom wrote: >> On current xen-unstable (4.14 to be) and AMD cpu: >> >> After rebooting the host, while the guests are starting, I hit the assertion >> below. >> xen-staging-4.13 seems fine on the same machine. > >

Re: [Xen-devel] [PATCH 2/3] arm64: remove uaccess_ttbr0 asm macros from cache functions

2019-12-04 Thread Pavel Tatashin
On Thu, Nov 28, 2019 at 9:51 AM Mark Rutland wrote: > > On Wed, Nov 27, 2019 at 01:44:52PM -0500, Pavel Tatashin wrote: > > We currently duplicate the logic to enable/disable uaccess via TTBR0, > > with C functions and assembly macros. This is a maintenenace burden > > and is liable to lead to

Re: [Xen-devel] [PATCH 4/4] x86/svm: Use named (bit)fields for task switch exit info

2019-12-04 Thread Andrew Cooper
On 04/12/2019 10:24, Jan Beulich wrote: > On 04.12.2019 10:43, Andrew Cooper wrote: >> Introduce vmcb.e1.* and vmcb.e2.* to provide names to fields in exitinfo{1,2} >> respectively. Implement the task switch names for now, and clean up the >> TASK_SWITCH handler. > "e1" and "e2" look overly short

Re: [Xen-devel] [PATCH 3/4] x86/svm: Clean up intinfo_t variables

2019-12-04 Thread Andrew Cooper
On 04/12/2019 09:43, Andrew Cooper wrote: > @@ -420,10 +420,10 @@ struct vmcb_struct { > u64 exitcode; /* offset 0x70 */ > u64 exitinfo1; /* offset 0x78 */ > u64 exitinfo2; /* offset 0x80 */ > -eventinj_t exitintinfo;/* offset 0x88 */

Re: [Xen-devel] [PATCH 3/4] x86/svm: Clean up intinfo_t variables

2019-12-04 Thread Andrew Cooper
On 04/12/2019 10:19, Jan Beulich wrote: > On 04.12.2019 10:43, Andrew Cooper wrote: >> --- a/xen/arch/x86/hvm/svm/intr.c >> +++ b/xen/arch/x86/hvm/svm/intr.c >> @@ -43,15 +43,13 @@ static void svm_inject_nmi(struct vcpu *v) >> { >> struct vmcb_struct *vmcb = v->arch.hvm.svm.vmcb; >> u32

Re: [Xen-devel] [PATCH 1/4] x86/svm: Clean up construct_vmcb()

2019-12-04 Thread Andrew Cooper
On 04/12/2019 10:06, Jan Beulich wrote: > On 04.12.2019 10:43, Andrew Cooper wrote: >> The vmcb is zeroed on allocate - drop all explicit writes of 0. Move >> hvm_update_guest_efer() to co-locate it with the other control register >> updates. >> >> Move the BUILD_BUG_ON() into build_assertions(),

Re: [Xen-devel] [PATCH v2 16/22] golang/xenlight: implement keyed union C to Go marshaling

2019-12-04 Thread George Dunlap
On 11/15/19 7:44 PM, Nick Rosbrook wrote: > From: Nick Rosbrook > > Switch over union key to determine how to populate 'union' in Go struct. > > Since the unions of C types cannot be directly accessed, add C structs in > cgo preamble to assist in marshaling keyed unions. This allows the C >

[Xen-devel] [xen-unstable test] 144520: tolerable FAIL

2019-12-04 Thread osstest service owner
flight 144520 xen-unstable real [real] http://logs.test-lab.xenproject.org/osstest/logs/144520/ Failures :-/ but no regressions. Tests which are failing intermittently (not blocking): test-armhf-armhf-xl-rtds 7 xen-boot fail in 144512 pass in 144520

Re: [Xen-devel] [PATCH v2 15/22] golang/xenlight: begin C to Go type marshaling

2019-12-04 Thread George Dunlap
On 11/15/19 7:44 PM, Nick Rosbrook wrote: > From: Nick Rosbrook > > Implement basic type conversion in fromC functions such as strings and > integer types. Also, remove existing toGo functions from xenlight.go in > favor of the new generated functions. You should probably say here explicitly

Re: [Xen-devel] [PATCH v4 3/9] x86/mm: introduce l{1, 2}t local variables to map_pages_to_xen

2019-12-04 Thread Xia, Hongyan
>> @@ -5272,6 +5279,7 @@ int map_pages_to_xen( >> ((1u << PAGETABLE_ORDER) - 1)) == 0)) ) >> { >> unsigned long base_mfn; >> +l1_pgentry_t *l1t; > > const, as this looks to be used for lookup only? I cannot do this for now since

Re: [Xen-devel] [PATCH 1/3] arm/arm64/xen: use C inlines for privcmd_call

2019-12-04 Thread Pavel Tatashin
On Fri, Nov 29, 2019 at 10:05 AM Julien Grall wrote: > > Hi, > > On 27/11/2019 18:44, Pavel Tatashin wrote: > > diff --git a/arch/arm64/include/asm/xen/hypercall.h > > b/arch/arm64/include/asm/xen/hypercall.h > > index 3522cbaed316..1a74fb28607f 100644 > > ---

Re: [Xen-devel] [PATCH 1/3] arm/arm64/xen: use C inlines for privcmd_call

2019-12-04 Thread Pavel Tatashin
On Fri, Nov 29, 2019 at 10:10 AM Andrew Cooper wrote: > > On 29/11/2019 15:05, Julien Grall wrote: > > Hi, > > > > On 27/11/2019 18:44, Pavel Tatashin wrote: > >> diff --git a/arch/arm64/include/asm/xen/hypercall.h > >> b/arch/arm64/include/asm/xen/hypercall.h > >> index

Re: [Xen-devel] [PATCH v4 2/9] x86: introduce a new set of APIs to manage Xen page tables

2019-12-04 Thread Xia, Hongyan
> There's no need for the map/unmap functions to have a _new > suffix, is there? I thought this was weird at first also, but what I find really useful is that we can just change all call sites to the new API in small steps without breaking. Otherwise we have to merge a huge batch of changes

Re: [Xen-devel] xen-unstable (4.14 to be): Assertion '!preempt_count()' failed at preempt.c:36

2019-12-04 Thread Jan Beulich
On 04.12.2019 18:21, Sander Eikelenboom wrote: > On current xen-unstable (4.14 to be) and AMD cpu: > > After rebooting the host, while the guests are starting, I hit the assertion > below. > xen-staging-4.13 seems fine on the same machine. Nothing between 4.13 RC4 and the tip of staging stands

Re: [Xen-devel] [PATCH v2 13/22] golang/xenlight: generate structs from the IDL

2019-12-04 Thread George Dunlap
On 11/15/19 7:44 PM, Nick Rosbrook wrote: > From: Nick Rosbrook > > Add struct and keyed union generation to gengotypes.py. For keyed unions, > use a method similar to gRPC's oneof to interpret C unions as Go types. > Meaning, for a given struct with a union field, generate a struct for > each

[Xen-devel] xen-unstable (4.14 to be): Assertion '!preempt_count()' failed at preempt.c:36

2019-12-04 Thread Sander Eikelenboom
L.S., On current xen-unstable (4.14 to be) and AMD cpu: After rebooting the host, while the guests are starting, I hit the assertion below. xen-staging-4.13 seems fine on the same machine. -- Sander (XEN) [2019-12-04 17:03:25.062] grant_table.c:1808:d7v0 Expanding d7 grant table from 3 to 4

[Xen-devel] [PATCH v4 9/9] x86/mm: change pl*e to l*t in virt_to_xen_l*e

2019-12-04 Thread Hongyan Xia
From: Wei Liu We will need to have a variable named pl*e when we rewrite virt_to_xen_l*e. Change pl*e to l*t to reflect better its purpose. This will make reviewing later patch easier. No functional change. Signed-off-by: Wei Liu Signed-off-by: Hongyan Xia Reviewed-by: Jan Beulich ---

[Xen-devel] [PATCH v4 0/9] Add alternative API for Xen PTEs

2019-12-04 Thread Hongyan Xia
NOTE: My email address has changed due to some HR management. I have lost all my previous emails and I could only salvage some of the comments to v3 from the mailing list archive. I will reply to the comments from v3 in this v4 series. This batch adds an alternative alloc-map-unmap-free Xen PTE

[Xen-devel] [PATCH v4 3/9] x86/mm: introduce l{1, 2}t local variables to map_pages_to_xen

2019-12-04 Thread Hongyan Xia
From: Wei Liu The pl2e and pl1e variables are heavily (ab)used in that function. It is fine at the moment because all page tables are always mapped so there is no need to track the life time of each variable. We will soon have the requirement to map and unmap page tables. We need to track the

[Xen-devel] [PATCH v4 4/9] x86/mm: introduce l{1, 2}t local variables to modify_xen_mappings

2019-12-04 Thread Hongyan Xia
From: Wei Liu The pl2e and pl1e variables are heavily (ab)used in that function. It is fine at the moment because all page tables are always mapped so there is no need to track the life time of each variable. We will soon have the requirement to map and unmap page tables. We need to track the

[Xen-devel] [PATCH v4 6/9] x86/mm: add an end_of_loop label in map_pages_to_xen

2019-12-04 Thread Hongyan Xia
From: Wei Liu We will soon need to clean up mappings whenever the out most loop is ended. Add a new label and turn relevant continue's into goto's. No functional change. Signed-off-by: Wei Liu --- xen/arch/x86/mm.c | 9 + 1 file changed, 5 insertions(+), 4 deletions(-) diff --git

[Xen-devel] [PATCH v4 8/9] x86/mm: add an end_of_loop label in modify_xen_mappings

2019-12-04 Thread Hongyan Xia
From: Wei Liu We will soon need to clean up mappings whenever the out most loop is ended. Add a new label and turn relevant continue's into goto's. No functional change. Signed-off-by: Wei Liu --- xen/arch/x86/mm.c | 15 --- 1 file changed, 8 insertions(+), 7 deletions(-) diff

[Xen-devel] [PATCH v4 1/9] x86: move some xen mm function declarations

2019-12-04 Thread Hongyan Xia
From: Wei Liu They were put into page.h but mm.h is more appropriate. The real reason is that I will be adding some new functions which takes mfn_t. It turns out it is a bit difficult to do in page.h. No functional change. Signed-off-by: Wei Liu Acked-by: Jan Beulich --- Changed since v3:

[Xen-devel] [PATCH v4 7/9] x86/mm: make sure there is one exit path for modify_xen_mappings

2019-12-04 Thread Hongyan Xia
From: Wei Liu We will soon need to handle dynamically mapping / unmapping page tables in the said function. No functional change. Signed-off-by: Wei Liu Signed-off-by: Hongyan Xia --- Changed since v3: - remove asserts on rc since it never gets changed to anything else. ---

[Xen-devel] [PATCH v4 5/9] x86/mm: map_pages_to_xen would better have one exit path

2019-12-04 Thread Hongyan Xia
From: Wei Liu We will soon rewrite the function to handle dynamically mapping and unmapping of page tables. No functional change. Signed-off-by: Wei Liu Signed-off-by: Hongyan Xia --- Changed since v3: - remove asserts on rc since rc never gets changed to anything else - reword commit

[Xen-devel] [PATCH v4 2/9] x86: introduce a new set of APIs to manage Xen page tables

2019-12-04 Thread Hongyan Xia
From: Wei Liu We are going to switch to using domheap page for page tables. A new set of APIs is introduced to allocate, map, unmap and free pages for page tables. The allocation and deallocation work on mfn_t but not page_info, because they are required to work even before frame table is set

Re: [Xen-devel] [PATCH v2 12/22] golang/xenlight: re-factor Hwcap type implementation

2019-12-04 Thread George Dunlap
On 11/15/19 7:44 PM, Nick Rosbrook wrote: > From: Nick Rosbrook > > Re-define Hwcap as [8]uint32, and implement toC function. Also, re-name and > modify signature of toGo function to fromC. > > Signed-off-by: Nick Rosbrook > Reviewed-by: George Dunlap > --- > Changes in v2: > - Fix comment in

Re: [Xen-devel] [PATCH v3] x86: do not enable global pages when virtualized on AMD hardware

2019-12-04 Thread Jan Beulich
On 04.12.2019 17:18, Roger Pau Monné wrote: > On Wed, Dec 04, 2019 at 05:11:42PM +0100, Jan Beulich wrote: >> On 04.12.2019 16:12, Roger Pau Monne wrote: >>> @@ -130,7 +143,7 @@ unsigned long pv_make_cr4(const struct vcpu *v) >>> */ >>> if ( d->arch.pv.pcid ) >>> cr4 |=

Re: [Xen-devel] [PATCH v2 11/22] golang/xenlight: re-factor Uuid type implementation

2019-12-04 Thread George Dunlap
On 11/15/19 7:44 PM, Nick Rosbrook wrote: > From: Nick Rosbrook > > Re-define Uuid as [16]byte and implement fromC, toC, and String functions. > > Signed-off-by: Nick Rosbrook > --- > tools/golang/xenlight/xenlight.go | 37 +-- > 1 file changed, 35 insertions(+), 2

Re: [Xen-devel] [PATCH v2 08/22] golang/xenlight: define MsVmGenid builtin type

2019-12-04 Thread George Dunlap
On 11/15/19 7:44 PM, Nick Rosbrook wrote: > From: Nick Rosbrook > > Define MsVmGenid as [int(C.LIBXL_MS_VM_GENID_LEN)]byte and implement fromC > and toC functions. > > Signed-off-by: Nick Rosbrook > Reviewed-by: George Dunlap > --- > tools/golang/xenlight/xenlight.go | 23

Re: [Xen-devel] [PATCH v2 10/22] golang/xenlight: define CpuidPolicyList builtin type

2019-12-04 Thread George Dunlap
On 11/15/19 7:44 PM, Nick Rosbrook wrote: > From: Nick Rosbrook > > Define CpuidPolicyList as a string so that libxl_cpuid_parse_config can > be used in the toC function. > > For now, fromC is a no-op since libxl does not support a way to read a > policy, modify it,and then give it back to

Re: [Xen-devel] [RFC v5 024/126] error: auto propagated local_err

2019-12-04 Thread Markus Armbruster
Vladimir Sementsov-Ogievskiy writes: > Here is introduced ERRP_AUTO_PROPAGATE macro, to be used at start of > functions with errp OUT parameter. > > It has three goals: > > 1. Fix issue with error_fatal & error_prepend/error_append_hint: user > can't see this additional information, because

Re: [Xen-devel] [PATCH] x86/debug: Plumb pending_dbg through the monitor and devicemodel interfaces

2019-12-04 Thread Petre Ovidiu PIRCALABU
On Tue, 2019-12-03 at 17:10 +, Andrew Cooper wrote: > Like %cr2 for pagefaults, %dr6 contains ancillary information for > debug > exceptions, and needs similar handling. > > For xendevicemodel_inject_event(), no ABI change is needed (although > an API > one would be ideal). Switch from 'cr2'

Re: [Xen-devel] [PATCH] x86/debug: Plumb pending_dbg through the monitor and devicemodel interfaces

2019-12-04 Thread Petre Ovidiu PIRCALABU
On Wed, 2019-12-04 at 10:04 +0100, Jan Beulich wrote: > On 03.12.2019 18:10, Andrew Cooper wrote: > > --- a/xen/include/public/hvm/dm_op.h > > +++ b/xen/include/public/hvm/dm_op.h > > @@ -324,7 +324,7 @@ struct xen_dm_op_inject_event { > > /* IN - error code (or ~0 to skip) */ > >

[Xen-devel] [PATCH v3 3/4] x86/smp: check APIC ID on AP bringup

2019-12-04 Thread Roger Pau Monne
Check that the processor to be woken up APIC ID is addressable in the current APIC mode. Note that in practice systems with APIC IDs > 255 should already have x2APIC enabled by the firmware, and hence this is mostly a safety belt. Signed-off-by: Roger Pau Monné --- Changes since v2: - Reword

[Xen-devel] [PATCH v3 2/4] x86/apic: force phys mode if interrupt remapping is disabled

2019-12-04 Thread Roger Pau Monne
Cluster mode can only be used with interrupt remapping support, since the top 16bits of the APIC ID are filled with the cluster ID, and hence on systems where the physical ID is still smaller than 255 the cluster ID is not. Force x2APIC to use physical mode if there's no interrupt remapping

[Xen-devel] [PATCH v3 1/4] x86/ioapic: only use dest32 with x2apic and interrupt remapping enabled

2019-12-04 Thread Roger Pau Monne
The IO-APIC code assumes that x2apic being enabled also implies interrupt remapping being enabled, and hence will use the 32bit destination field in the IO-APIC entry. This is safe now, but there's no reason to not enable x2APIC even without interrupt remapping, and hence the IO-APIC code needs

[Xen-devel] [PATCH v3 0/4] x86: enable x2APIC mode regardless of interrupt remapping support

2019-12-04 Thread Roger Pau Monne
Hello, The following series aims to allow enabling x2APIC mode without interrupt remapping support. The main usage of this would be in virtualized environments, that usually provide x2APIC support but not interrupt remapping. See the last patch for some performance numbers of using x2APIC over

[Xen-devel] [PATCH v3 4/4] x86/apic: allow enabling x2APIC mode regardless of interrupt remapping

2019-12-04 Thread Roger Pau Monne
x2APIC mode doesn't mandate interrupt remapping, and hence can be enabled independently. This patch enables x2APIC when available, regardless of whether there's interrupt remapping support. This is beneficial specially when running on virtualized environments, since it reduces the amount of

Re: [Xen-devel] [PATCH v3] x86: do not enable global pages when virtualized on AMD hardware

2019-12-04 Thread Roger Pau Monné
On Wed, Dec 04, 2019 at 05:11:42PM +0100, Jan Beulich wrote: > On 04.12.2019 16:12, Roger Pau Monne wrote: > > --- a/xen/arch/x86/pv/domain.c > > +++ b/xen/arch/x86/pv/domain.c > > @@ -118,6 +118,19 @@ unsigned long pv_fixup_guest_cr4(const struct vcpu *v, > > unsigned long cr4) > >

Re: [Xen-devel] [PATCH v2 07/22] golang/xenlight: define Mac builtin type

2019-12-04 Thread George Dunlap
On 11/15/19 7:44 PM, Nick Rosbrook wrote: > From: Nick Rosbrook > > Define Mac as [6]byte and implement fromC, toC, and String functions. > > Signed-off-by: Nick Rosbrook Reviewed-by: George Dunlap ___ Xen-devel mailing list

Re: [Xen-devel] [PATCH v2 06/22] golang/xenlight: define StringList builtin type

2019-12-04 Thread George Dunlap
On 11/15/19 7:44 PM, Nick Rosbrook wrote: > From: Nick Rosbrook > > Define StringList as []string an implement fromC and toC functions. > > Signed-off-by: Nick Rosbrook Reviewed-by: George Dunlap ___ Xen-devel mailing list

Re: [Xen-devel] [PATCH v3] xen/x86: vpmu: Unmap per-vCPU PMU page when the domain is destroyed

2019-12-04 Thread Julien Grall
On 04/12/2019 16:08, Jan Beulich wrote: On 04.12.2019 17:02, Julien Grall wrote: On 04/12/2019 15:59, Boris Ostrovsky wrote: On Dec 4, 2019, at 10:55 AM, Julien Grall wrote: Hi Boris, On 28/11/2019 21:50, Boris Ostrovsky wrote: On 11/28/19 5:23 AM, Jan Beulich wrote: On 28.11.2019

Re: [Xen-devel] [PATCH v3] x86: do not enable global pages when virtualized on AMD hardware

2019-12-04 Thread Jan Beulich
On 04.12.2019 16:12, Roger Pau Monne wrote: > --- a/xen/arch/x86/pv/domain.c > +++ b/xen/arch/x86/pv/domain.c > @@ -118,6 +118,19 @@ unsigned long pv_fixup_guest_cr4(const struct vcpu *v, > unsigned long cr4) > (mmu_cr4_features & PV_CR4_GUEST_VISIBLE_MASK)); > } > > +static

Re: [Xen-devel] [PATCH v2 04/22] golang/xenlight: define KeyValueList as empty struct

2019-12-04 Thread George Dunlap
On 11/15/19 7:44 PM, Nick Rosbrook wrote: > From: Nick Rosbrook > > Define KeyValueList as empty struct as there is currently no reason for > this type to be available in the Go package. > > Implement fromC and toC functions as no-ops. > > Signed-off-by: Nick Rosbrook Reviewed-by: George

Re: [Xen-devel] [PATCH v3] xen/x86: vpmu: Unmap per-vCPU PMU page when the domain is destroyed

2019-12-04 Thread Jan Beulich
On 04.12.2019 17:02, Julien Grall wrote: > > > On 04/12/2019 15:59, Boris Ostrovsky wrote: >> >> >>> On Dec 4, 2019, at 10:55 AM, Julien Grall wrote: >>> >>> Hi Boris, >>> >>> On 28/11/2019 21:50, Boris Ostrovsky wrote: On 11/28/19 5:23 AM, Jan Beulich wrote: > On 28.11.2019 10:38,

Re: [Xen-devel] [PATCH v3] xen/x86: vpmu: Unmap per-vCPU PMU page when the domain is destroyed

2019-12-04 Thread Julien Grall
On 04/12/2019 15:59, Boris Ostrovsky wrote: On Dec 4, 2019, at 10:55 AM, Julien Grall wrote: Hi Boris, On 28/11/2019 21:50, Boris Ostrovsky wrote: On 11/28/19 5:23 AM, Jan Beulich wrote: On 28.11.2019 10:38, Paul Durrant wrote: --- a/xen/arch/x86/cpu/vpmu.c +++

Re: [Xen-devel] [PATCH v3] xen/x86: vpmu: Unmap per-vCPU PMU page when the domain is destroyed

2019-12-04 Thread Boris Ostrovsky
> On Dec 4, 2019, at 10:55 AM, Julien Grall wrote: > > Hi Boris, > > On 28/11/2019 21:50, Boris Ostrovsky wrote: >> On 11/28/19 5:23 AM, Jan Beulich wrote: >>> On 28.11.2019 10:38, Paul Durrant wrote: >>> --- a/xen/arch/x86/cpu/vpmu.c +++ b/xen/arch/x86/cpu/vpmu.c @@ -576,11

Re: [Xen-devel] [PATCH v2 01/22] golang/xenlight: generate enum types from IDL

2019-12-04 Thread George Dunlap
On 11/15/19 7:44 PM, Nick Rosbrook wrote: > From: Nick Rosbrook > > Introduce gengotypes.py to generate Go code the from IDL. As a first step, > implement 'enum' type generation. > > As a result of the newly-generated code, remove the existing, and now > conflicting definitions in xenlight.go.

Re: [Xen-devel] vmi: supporting single-stepping on AMD processors

2019-12-04 Thread Andrew Cooper
On 04/12/2019 15:32, felix nasch wrote: > Hi, > > I am working on making libvmi[1] work on AMD, and one big issue is the lack > of support for single-stepping on AMD processors in the vmi api offered by > xen. > I think i have a way to get it to work, and I would like to know if you see > any

Re: [Xen-devel] [PATCH v3] xen/x86: vpmu: Unmap per-vCPU PMU page when the domain is destroyed

2019-12-04 Thread Julien Grall
Hi Boris, On 28/11/2019 21:50, Boris Ostrovsky wrote: On 11/28/19 5:23 AM, Jan Beulich wrote: On 28.11.2019 10:38, Paul Durrant wrote: --- a/xen/arch/x86/cpu/vpmu.c +++ b/xen/arch/x86/cpu/vpmu.c @@ -576,11 +576,36 @@ static void vpmu_arch_destroy(struct vcpu *v)

Re: [Xen-devel] Ad-hoc technical community meeting Dec 10 pm and 11 am in Cambridge, hosted @ Citrix

2019-12-04 Thread George Dunlap
On 12/2/19 6:49 PM, Lars Kurth wrote: > If you want to attend, please let me know. > > I arranged a room for the meeting and will be there on the 10th (I cannot do > the 11th). There will be no food provided during the day, but of course there > are snacks and coffee in our kitchen. > > I was

Re: [Xen-devel] [PATCH v2 02/22] golang/xenlight: define Defbool builtin type

2019-12-04 Thread George Dunlap
On 11/15/19 7:44 PM, Nick Rosbrook wrote: > From: Nick Rosbrook > > Define Defbool as struct analagous to the C type, and define the type > 'defboolVal' that represent true, false, and default defbool values. > > Implement Set, Unset, SetIfDefault, IsDefault, Val, and String functions > on

[Xen-devel] vmi: supporting single-stepping on AMD processors

2019-12-04 Thread felix nasch
Hi, I am working on making libvmi[1] work on AMD, and one big issue is the lack of support for single-stepping on AMD processors in the vmi api offered by xen. I think i have a way to get it to work, and I would like to know if you see any issues with this approach, and if there is any reason it

[Xen-devel] [PATCH v3] x86: do not enable global pages when virtualized on AMD hardware

2019-12-04 Thread Roger Pau Monne
When using global pages a full tlb flush can only be performed by toggling the PGE bit in CR4, which is usually quite expensive in terms of performance when running virtualized. This is specially relevant on AMD hardware, which doesn't have the ability to do selective CR4 trapping, but can also be

Re: [Xen-devel] [PATCH livepatch-build-tools] create-diff-object: Ignore .init sections

2019-12-04 Thread Ross Lagerwall
On 12/3/19 7:57 AM, Pawel Wieczorkiewicz wrote: > The .init sections must not be considered for patching regardless of > whether they are CHANGED or NEW. > Explicitely detect and ignore all such sections, before marking > ignored sections as SAME. > > Signed-off-by: Pawel Wieczorkiewicz

Re: [Xen-devel] [PATCH livepatch-build-tools] create-diff-object: Include string sections later

2019-12-04 Thread Ross Lagerwall
On 12/3/19 7:57 AM, Pawel Wieczorkiewicz wrote: > ... when all symbols have their status and include flags processed. > > Processing special sections may include additional symbols. String > sections (.rodata*) are included iff they are referenced by at least > one symbol. Thus, in order to

Re: [Xen-devel] [PATCH v2 4/4] x86/apic: allow enabling x2APIC mode regardless of interrupt remapping

2019-12-04 Thread Jan Beulich
On 04.12.2019 14:51, Roger Pau Monné wrote: > On Tue, Dec 03, 2019 at 04:33:09PM +0100, Jan Beulich wrote: >> On 29.11.2019 12:28, Roger Pau Monne wrote: >>> --- a/xen/arch/x86/apic.c >>> +++ b/xen/arch/x86/apic.c >>> @@ -492,7 +492,8 @@ static void __enable_x2apic(void) >>> >>> static void

Re: [Xen-devel] [PATCH v2] x86: do not enable global pages when virtualized on AMD hardware

2019-12-04 Thread Jan Beulich
On 04.12.2019 14:17, Roger Pau Monné wrote: > On Wed, Dec 04, 2019 at 02:15:58PM +0100, Jan Beulich wrote: >> On 04.12.2019 12:52, Roger Pau Monné wrote: >>> On Wed, Dec 04, 2019 at 12:05:35PM +0100, Jan Beulich wrote: On 04.12.2019 11:44, Roger Pau Monne wrote: > ---

Re: [Xen-devel] [PATCH v2 4/4] x86/apic: allow enabling x2APIC mode regardless of interrupt remapping

2019-12-04 Thread Roger Pau Monné
On Tue, Dec 03, 2019 at 04:33:09PM +0100, Jan Beulich wrote: > On 29.11.2019 12:28, Roger Pau Monne wrote: > > --- a/xen/arch/x86/apic.c > > +++ b/xen/arch/x86/apic.c > > @@ -492,7 +492,8 @@ static void __enable_x2apic(void) > > > > static void resume_x2apic(void) > > { > > -

Re: [Xen-devel] bug: unable to LZ4 decompress ub1910 installer kernel when launching domU

2019-12-04 Thread Jan Beulich
On 04.12.2019 08:14, Jeremi Piotrowski wrote: > Any suggestions how to proceed? Actually here's a better version (I think). Jan lz4: refine commit 9143a6c55ef7 for the 64-bit case I clearly went too far there: While the LZ4_WILDCOPY() instances indeed need prior guarding, LZ4_SECURECOPY()

Re: [Xen-devel] [PATCH v2] x86: do not enable global pages when virtualized on AMD hardware

2019-12-04 Thread Roger Pau Monné
On Wed, Dec 04, 2019 at 02:15:58PM +0100, Jan Beulich wrote: > On 04.12.2019 12:52, Roger Pau Monné wrote: > > On Wed, Dec 04, 2019 at 12:05:35PM +0100, Jan Beulich wrote: > >> On 04.12.2019 11:44, Roger Pau Monne wrote: > >>> --- a/xen/arch/x86/pv/domain.c > >>> +++ b/xen/arch/x86/pv/domain.c >

Re: [Xen-devel] [PATCH v2] x86: do not enable global pages when virtualized on AMD hardware

2019-12-04 Thread Jan Beulich
On 04.12.2019 12:52, Roger Pau Monné wrote: > On Wed, Dec 04, 2019 at 12:05:35PM +0100, Jan Beulich wrote: >> On 04.12.2019 11:44, Roger Pau Monne wrote: >>> --- a/xen/arch/x86/pv/domain.c >>> +++ b/xen/arch/x86/pv/domain.c >>> @@ -118,11 +118,18 @@ unsigned long pv_fixup_guest_cr4(const struct

Re: [Xen-devel] bug: unable to LZ4 decompress ub1910 installer kernel when launching domU

2019-12-04 Thread Jan Beulich
On 04.12.2019 08:14, Jeremi Piotrowski wrote: > I'm able to repro, and I isolated the code from xc_dom_bzimageloader.c, > xc_dom_decompress_lz4.c and /xen/common/lz4/decompress.c such that I can > test more easily (I'm using code from 4.12.1). I'm testing with > vmlinuz-5.3.0-23-generic installed

[Xen-devel] [xen-unstable-smoke test] 144521: tolerable all pass - PUSHED

2019-12-04 Thread osstest service owner
flight 144521 xen-unstable-smoke real [real] http://logs.test-lab.xenproject.org/osstest/logs/144521/ Failures :-/ but no regressions. Tests which did not succeed, but are not blocking: test-amd64-amd64-libvirt 13 migrate-support-checkfail never pass test-arm64-arm64-xl-xsm

Re: [Xen-devel] [PATCH 0/2] xen/blkback: Aggressively shrink page pools if a memory pressure is detected

2019-12-04 Thread sjpark
On 04.12.19 12:52, Durrant, Paul wrote: >> -Original Message- >> From: Xen-devel On Behalf Of >> SeongJae Park >> Sent: 04 December 2019 11:34 >> To: konrad.w...@oracle.com; roger@citrix.com; ax...@kernel.dk >> Cc: sj38.p...@gmail.com; xen-devel@lists.xenproject.org; linux- >>

Re: [Xen-devel] [PATCH v2] x86: do not enable global pages when virtualized on AMD hardware

2019-12-04 Thread Roger Pau Monné
On Wed, Dec 04, 2019 at 12:05:35PM +0100, Jan Beulich wrote: > On 04.12.2019 11:44, Roger Pau Monne wrote: > > When using global pages a full tlb flush can only be performed by > > toggling the PGE bit in CR4, which is usually quite expensive in terms > > of performance when running virtualized.

Re: [Xen-devel] [PATCH 0/2] xen/blkback: Aggressively shrink page pools if a memory pressure is detected

2019-12-04 Thread Durrant, Paul
> -Original Message- > From: Xen-devel On Behalf Of > SeongJae Park > Sent: 04 December 2019 11:34 > To: konrad.w...@oracle.com; roger@citrix.com; ax...@kernel.dk > Cc: sj38.p...@gmail.com; xen-devel@lists.xenproject.org; linux- > bl...@vger.kernel.org; linux-ker...@vger.kernel.org;

[Xen-devel] [PATCH 2/2] blkback: Add a module parameter for aggressive pool shrinking duration

2019-12-04 Thread SeongJae Park
From: SeongJae Park As discussed by the previous commit ("xen/blkback: Aggressively shrink page pools if a memory pressure is detected"), the aggressive pool shrinking duration should be carefully selected: ``If it is too long, free pages pool shrinking overhead can reduce the I/O performance.

[Xen-devel] [PATCH 1/2] xen/blkback: Aggressively shrink page pools if a memory pressure is detected

2019-12-04 Thread SeongJae Park
From: SeongJae Park Each `blkif` has a free pages pool for the grant mapping. The size of the pool starts from zero and be increased on demand while processing the I/O requests. If current I/O requests handling is finished or 100 milliseconds has passed since last I/O requests handling, it

[Xen-devel] [PATCH 0/2] xen/blkback: Aggressively shrink page pools if a memory pressure is detected

2019-12-04 Thread SeongJae Park
Each `blkif` has a free pages pool for the grant mapping. The size of the pool starts from zero and be increased on demand while processing the I/O requests. If current I/O requests handling is finished or 100 milliseconds has passed since last I/O requests handling, it checks and shrinks the

Re: [Xen-devel] [PATCH v2] x86: do not enable global pages when virtualized on AMD hardware

2019-12-04 Thread Jan Beulich
On 04.12.2019 11:44, Roger Pau Monne wrote: > When using global pages a full tlb flush can only be performed by > toggling the PGE bit in CR4, which is usually quite expensive in terms > of performance when running virtualized. This is specially relevant on > AMD hardware, which doesn't have the

[Xen-devel] [qemu-mainline test] 144514: tolerable FAIL - PUSHED

2019-12-04 Thread osstest service owner
flight 144514 qemu-mainline real [real] http://logs.test-lab.xenproject.org/osstest/logs/144514/ Failures :-/ but no regressions. Tests which did not succeed, but are not blocking: test-amd64-amd64-xl-rtds 18 guest-localmigrate/x10 fail like 144495

Re: [Xen-devel] [PATCH v2 2/4] x86/apic: force phys mode if interrupt remapping is disabled

2019-12-04 Thread Roger Pau Monné
On Wed, Dec 04, 2019 at 10:34:14AM +0100, Jan Beulich wrote: > On 04.12.2019 10:17, Roger Pau Monné wrote: > > On Tue, Dec 03, 2019 at 04:14:46PM +0100, Jan Beulich wrote: > >> On 29.11.2019 12:38, Roger Pau Monné wrote: > >>> On Fri, Nov 29, 2019 at 12:28:49PM +0100, Roger Pau Monne wrote: >

Re: [Xen-devel] [PATCH v2 1/4] x86/ioapic: only use dest32 with x2apic and interrupt remapping enabled

2019-12-04 Thread Roger Pau Monné
On Tue, Dec 03, 2019 at 04:11:07PM +0100, Jan Beulich wrote: > On 29.11.2019 12:28, Roger Pau Monne wrote: > > --- a/xen/arch/x86/io_apic.c > > +++ b/xen/arch/x86/io_apic.c > > @@ -562,7 +562,7 @@ set_ioapic_affinity_irq(struct irq_desc *desc, const > > cpumask_t *mask) > > > > dest =

[Xen-devel] [PATCH v2] x86: do not enable global pages when virtualized on AMD hardware

2019-12-04 Thread Roger Pau Monne
When using global pages a full tlb flush can only be performed by toggling the PGE bit in CR4, which is usually quite expensive in terms of performance when running virtualized. This is specially relevant on AMD hardware, which doesn't have the ability to do selective CR4 trapping, but can also be

[Xen-devel] [ovmf test] 144518: all pass - PUSHED

2019-12-04 Thread osstest service owner
flight 144518 ovmf real [real] http://logs.test-lab.xenproject.org/osstest/logs/144518/ Perfect :-) All tests in this flight passed as required version targeted for testing: ovmf c8ff8e05afb6a20b1ae66aa80bb8636b664be0b2 baseline version: ovmf

Re: [Xen-devel] [PATCH v3 2/2] xen-blkback: allow module to be cleanly unloaded

2019-12-04 Thread Jürgen Groß
On 02.12.19 12:41, Paul Durrant wrote: Add a module_exit() to perform the necessary clean-up. Signed-off-by: Paul Durrant Reviewed-by: "Roger Pau Monné" Reviewed-by: Juergen Gross Pushed to xen/tip.git for-linus-5.5b Juergen ___ Xen-devel

Re: [Xen-devel] [PATCH v3 1/2] xen/xenbus: reference count registered modules

2019-12-04 Thread Jürgen Groß
On 02.12.19 12:41, Paul Durrant wrote: To prevent a PV driver module being removed whilst attached to its other end, and hence xenbus calling into potentially invalid text, take a reference on the module before calling the probe() method (dropping it if unsuccessful) and drop the reference after

Re: [Xen-devel] [PATCH 4/4] x86/svm: Use named (bit)fields for task switch exit info

2019-12-04 Thread Jan Beulich
On 04.12.2019 10:43, Andrew Cooper wrote: > Introduce vmcb.e1.* and vmcb.e2.* to provide names to fields in exitinfo{1,2} > respectively. Implement the task switch names for now, and clean up the > TASK_SWITCH handler. "e1" and "e2" look overly short - and hence possibly ambiguous - to me. Make

[Xen-devel] [libvirt test] 144517: tolerable all pass - PUSHED

2019-12-04 Thread osstest service owner
flight 144517 libvirt real [real] http://logs.test-lab.xenproject.org/osstest/logs/144517/ Failures :-/ but no regressions. Tests which did not succeed, but are not blocking: test-armhf-armhf-libvirt 14 saverestore-support-checkfail like 144501 test-armhf-armhf-libvirt-raw 13

  1   2   >