[PATCH] tools/python: Pass linker to Python build process

2020-09-11 Thread Elliott Mitchell
Unexpectedly the environment variable which needs to be passed is $LDSHARED and not $LD. Otherwise Python may find the build `ld` instead of the host `ld`. Replace $(LDFLAGS) with $(SHLIB_LDFLAGS) as Python needs shared objects it can load at runtime, not executables. This uses $(CC) instead of

Re: Adopting the Linux Kernel Memory Model in Xen?

2020-09-11 Thread Andrew Cooper
On 11/09/2020 17:33, Julien Grall wrote: > Hi all, > > At the moment, Xen doesn't have a formal memory model. Instead, we are > relying on intuitions. This can lead to heated discussion on what can > a processor/compiler do or not. > > We also have some helpers that nearly do the same (such as >

Re: [PATCH] tools/libs/stat: fix broken build

2020-09-11 Thread Andrew Cooper
On 11/09/2020 18:58, Wei Liu wrote: > On Fri, Sep 11, 2020 at 03:15:28PM +0200, Juergen Gross wrote: >> Making getBridge() static triggered a build error with some gcc versions: >> >> error: 'strncpy' output may be truncated copying 15 bytes from a string of >> length 255

[PATCH] tools: Delete XEN_DOMCTL_disable_migrate

2020-09-11 Thread Andrew Cooper
It is conceptually wrong for this information to exist in the hypervisor in the first place. Only the toolstack is capable of correctly reasoning about the non-migrateability of guests. This hypercall has only ever existed to control the visibility of the Invariant TSC flag to the guest. Now

[linux-linus test] 154104: regressions - FAIL

2020-09-11 Thread osstest service owner
flight 154104 linux-linus real [real] http://logs.test-lab.xenproject.org/osstest/logs/154104/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-boot fail REGR. vs. 152332

Re: [PATCH v3 00/11] Fix PM hibernation in Xen guests

2020-09-11 Thread Anchal Agarwal
On Fri, Aug 28, 2020 at 06:39:45PM +, Anchal Agarwal wrote: > On Fri, Aug 28, 2020 at 08:29:24PM +0200, Rafael J. Wysocki wrote: > > CAUTION: This email originated from outside of the organization. Do not > > click links or open attachments unless you can confirm the sender and know > > the

[xen-unstable-smoke test] 154129: tolerable all pass - PUSHED

2020-09-11 Thread osstest service owner
flight 154129 xen-unstable-smoke real [real] http://logs.test-lab.xenproject.org/osstest/logs/154129/ Failures :-/ but no regressions. Tests which did not succeed, but are not blocking: test-amd64-amd64-libvirt 13 migrate-support-checkfail never pass test-arm64-arm64-xl-xsm

Re: [PATCH 5/5] x86/pv: Simplify emulation for the 64bit base MSRs

2020-09-11 Thread Andrew Cooper
On 11/09/2020 11:01, Jan Beulich wrote: > On 09.09.2020 11:59, Andrew Cooper wrote: >> is_pv_32bit_domain() is an expensive predicate, but isn't used for >> speculative >> safety in this case. Swap to checking the Long Mode bit in the CPUID policy, >> which is the architecturally correct

Re: [PATCH] xen/hypfs: fix writing of custom parameter

2020-09-11 Thread Andrew Cooper
On 11/09/2020 13:28, Jürgen Groß wrote: > On 11.09.20 14:14, Andrew Cooper wrote: >> On 11/09/2020 06:30, Juergen Gross wrote: >>> Today the maximum allowed data length for writing a hypfs node is >>> tested in the generic hypfs_write() function. For custom runtime >>> parameters this might be

Re: [PATCH] tools/libs/stat: fix broken build

2020-09-11 Thread Wei Liu
On Fri, Sep 11, 2020 at 03:15:28PM +0200, Juergen Gross wrote: > Making getBridge() static triggered a build error with some gcc versions: > > error: 'strncpy' output may be truncated copying 15 bytes from a string of > length 255 [-Werror=stringop-truncation] > > Fix that by printing a sane

Re: [PATCH 3/8] Convert code-of-conduct to rst

2020-09-11 Thread George Dunlap
> On Sep 11, 2020, at 2:39 PM, Jürgen Groß wrote: > > On 11.09.20 14:40, George Dunlap wrote: >> >> +Conduct Team members >> + >> + >> Conduct Team members are project leadership team members from any >> sub-project. The current list of Conduct Team members is: >> + >>

Re: [PATCH 01/11 v2] gitignore: Move ignores from global to subdirectories

2020-09-11 Thread Elliott Mitchell
On Fri, Sep 11, 2020 at 08:29:51AM +0200, Jan Beulich wrote: > On 10.09.2020 20:22, Elliott Mitchell wrote: > > I had *thought* "./" would restrict to capturing files in the current > > directory, but after some testing and then some reading of the > > documentation (oh, `git check-ignore` is a

[qemu-mainline test] 154096: regressions - FAIL

2020-09-11 Thread osstest service owner
flight 154096 qemu-mainline real [real] http://logs.test-lab.xenproject.org/osstest/logs/154096/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-amd64-xl-qcow210 debian-di-installfail REGR. vs. 152631

Re: [PATCH 4/5] x86/pv: Optimise to the segment context switching paths

2020-09-11 Thread Andrew Cooper
On 11/09/2020 10:49, Jan Beulich wrote: > On 09.09.2020 11:59, Andrew Cooper wrote: >> Save the segment selectors with explicit asm, rather than with read_sreg(). >> This permits the use of MOV's m16 encoding, which avoids indirecting the >> selector value through a register. > Instead of this,

Re: [PATCH 0/8] Finding a home for the Code of Conduct

2020-09-11 Thread Stefano Stabellini
On Fri, 11 Sep 2020, George Dunlap wrote: > The Code of Conduct has been approved [1]; now we need to find it a > home. Since we've started using sphinx for the hypervisor documents, > I propose doing the same for the project-wide governance documents, starting > with the Code of Conduct. > >

[PATCH 6/8] Convert communication-practice.md to RST

2020-09-11 Thread George Dunlap
Convert title / sections, list formatting, and references as appropriate. Convert >-based blockquotes to RST-style block quotes. Apply the following style manually: * Each quote should be a separate block quote para * Quotes should be italicized. Labels ("BAD:") will be bold. For the

[PATCH 4/8] Convert communication-guide.md to rst

2020-09-11 Thread George Dunlap
Convert headers and add necessary spaces to recognize lists. Use the :doc: tag to cross-reference documents. NB that this will throw warnings for the documents not yet converted. No textual changes. Signed-off-by: George Dunlap --- source/code-of-conduct.rst| 4 +--

[PATCH 0/8] Finding a home for the Code of Conduct

2020-09-11 Thread George Dunlap
The Code of Conduct has been approved [1]; now we need to find it a home. Since we've started using sphinx for the hypervisor documents, I propose doing the same for the project-wide governance documents, starting with the Code of Conduct. This series takes Lars' code of conduct tree, written as

[PATCH 8/8] Clean up index page

2020-09-11 Thread George Dunlap
Get rid of "change this here" comments. Break into two sections: "Regulations", which are rules, and "Guides", which are suggestions. Remove unused indexes. Signed-off-by: George Dunlap --- source/index.rst | 26 +++--- 1 file changed, 19 insertions(+), 7 deletions(-)

[PATCH 5/8] Convert code-review-guide.md to rst

2020-09-11 Thread George Dunlap
Convert titles as approproate. Use inter-doc references for other full docs. Convert other external links to RST-style references, keeping the labels (3-F). One exception to this: sphinx noticed that there were two 'D' labels; rename one to `Shift Left`. Convert internal link to RST-style

[PATCH 1/8] Add basic sphinx structure

2020-09-11 Thread George Dunlap
--- Makefile | 20 +++ source/conf.py | 52 source/index.rst | 20 +++ 3 files changed, 92 insertions(+) create mode 100644 Makefile create mode 100644 source/conf.py create mode 100644 source/index.rst

[PATCH 3/8] Convert code-of-conduct to rst

2020-09-11 Thread George Dunlap
Underline section titles. Convert links to RST-style links. NB that the Communication Guide link won't work ATM; this will be fixed when we convert that document to RST. Adjust the formatting for the list so that it's converted properly. A couple of clean-ups: * Expand the label for

[PATCH 2/8] Move docs into source directory, rename to rst

2020-09-11 Thread George Dunlap
--- code-of-conduct.md => source/code-of-conduct.md | 0 code-review-guide.md => source/code-review-guide.md | 0 communication-guide.md => source/communication-guide.md | 0 communication-practice.md => source/communication-practice.md | 0 source/index.rst

Adopting the Linux Kernel Memory Model in Xen?

2020-09-11 Thread Julien Grall
Hi all, At the moment, Xen doesn't have a formal memory model. Instead, we are relying on intuitions. This can lead to heated discussion on what can a processor/compiler do or not. We also have some helpers that nearly do the same (such as {read,write}_atomic() vs ACCESS_ONCE()) with no

[xen-unstable-smoke test] 154120: tolerable all pass - PUSHED

2020-09-11 Thread osstest service owner
flight 154120 xen-unstable-smoke real [real] http://logs.test-lab.xenproject.org/osstest/logs/154120/ Failures :-/ but no regressions. Tests which did not succeed, but are not blocking: test-amd64-amd64-libvirt 13 migrate-support-checkfail never pass test-arm64-arm64-xl-xsm

Re: [PATCH] xen/hypfs: fix writing of custom parameter

2020-09-11 Thread Andrew Cooper
On 11/09/2020 06:30, Juergen Gross wrote: > Today the maximum allowed data length for writing a hypfs node is > tested in the generic hypfs_write() function. For custom runtime > parameters this might be wrong, as the maximum allowed size is derived > from the buffer holding the current setting,

[PATCH v2] x86: guard against straight-line speculation past RET

2020-09-11 Thread Jan Beulich
Under certain conditions CPUs can speculate into the instruction stream past a RET instruction. Guard against this just like 3b7dab93f240 ("x86/spec-ctrl: Protect against CALL/JMP straight-line speculation") did - by inserting an "INT $3" insn. It's merely the mechanics of how to achieve this that

[PATCH v2 0/2] Introduce cmpxchg64() and guest_cmpxchg64()

2020-09-11 Thread Julien Grall
From: Julien Grall Hi all, This small series introduced cmpxch64() and guest_cmpxchg64(). This will be useful when porting IOREQ server to Arm. Cheers, Julien Grall (2): xen/arm: Remove cmpxchg_local() and drop _mb from the other helpers xen: Introduce cmpxchg64() and guest_cmpxchg64()

[PATCH v2 1/2] xen/arm: Remove cmpxchg_local() and drop _mb from the other helpers

2020-09-11 Thread Julien Grall
From: Julien Grall The current set of helpers are quite confusing to follow as the naming is not very consistent. Given that cmpxchg_local() is not used in Xen, drop it completely. Furthermore, adopt a naming with _mb so all names are now consistent. Signed-off-by: Julien Grall ---

[PATCH v2 2/2] xen: Introduce cmpxchg64() and guest_cmpxchg64()

2020-09-11 Thread Julien Grall
From: Julien Grall The IOREQ code is using cmpxchg() with 64-bit value. At the moment, this is x86 code, but there is plan to make it common. To cater 32-bit arch, introduce two new helpers to deal with 64-bit cmpxchg(). The Arm 32-bit implementation of cmpxchg64() is based on the __cmpxchg64

Re: [PATCH] x86/PV: make post-migration page state consistent

2020-09-11 Thread Andrew Cooper
On 11/09/2020 11:34, Jan Beulich wrote: > When a page table page gets de-validated, its type reference count drops > to zero (and PGT_validated gets cleared), but its type remains intact. > XEN_DOMCTL_getpageframeinfo3, therefore, so far reported prior usage for > such pages. An intermediate write

Re: [PATCH] x86/PV: fold exit paths of ptwr_do_page_fault()

2020-09-11 Thread Andrew Cooper
On 11/09/2020 11:35, Jan Beulich wrote: > One less aspect to keep an eye on for things to stay in sync. > > Signed-off-by: Jan Beulich Acked-by: Andrew Cooper

Re: preparations for 4.13.2 and 4.12.4

2020-09-11 Thread Bertrand Marquis
> On 11 Sep 2020, at 15:25, Julien Grall wrote: > > Hi Bertrand, > > On 11/09/2020 14:56, Bertrand Marquis wrote: >>> On 11 Sep 2020, at 14:51, Julien Grall wrote: >>> >>> Hi Bertrand, >>> >>> On 11/09/2020 14:32, Bertrand Marquis wrote: > On 11 Sep 2020, at 14:11, Jan Beulich wrote:

Re: [PATCH v3 00/11] Fix PM hibernation in Xen guests

2020-09-11 Thread boris . ostrovsky
On 8/21/20 6:22 PM, Anchal Agarwal wrote: > > Known issues: > 1.KASLR causes intermittent hibernation failures. VM fails to resumes and > has to be restarted. I will investigate this issue separately and shouldn't > be a blocker for this patch series. Is there any change in status for this?

Re: [PATCH 2/2] xen/gntdev.c: Convert get_user_pages*() to pin_user_pages*()

2020-09-11 Thread boris . ostrovsky
On 9/6/20 2:51 AM, Souptick Joarder wrote: > In 2019, we introduced pin_user_pages*() and now we are converting > get_user_pages*() to the new API as appropriate. [1] & [2] could > be referred for more information. This is case 5 as per document [1]. > > [1]

[PATCH v3] efi: Always map EfiRuntimeServicesCode and EfiRuntimeServicesData

2020-09-11 Thread Sergey Temerkhanov
This helps overcome problems observed with some UEFI implementations which don't set the Attributes field in memery descriptors properly Signed-off-by: Sergey Temerkhanov --- xen/common/efi/boot.c| 27 ++- xen/include/efi/efidef.h | 6 ++ 2 files changed, 32

Re: [PATCH 1/2] xen/gntdev.c: Mark pages as dirty

2020-09-11 Thread boris . ostrovsky
On 9/6/20 2:51 AM, Souptick Joarder wrote: > There seems to be a bug in the original code when gntdev_get_page() > is called with writeable=true then the page needs to be marked dirty > before being put. > > To address this, a bool writeable is added in gnt_dev_copy_batch, set > it in

Re: preparations for 4.13.2 and 4.12.4

2020-09-11 Thread Julien Grall
On 11/09/2020 14:11, Jan Beulich wrote: All, the releases are about due, but will of course want to wait for the XSA fixes going public on the 22nd. Please point out backports you find missing from the respective staging branches, but which you consider relevant. (Ian, Julien, Stefano: I

Re: preparations for 4.13.2 and 4.12.4

2020-09-11 Thread Julien Grall
Hi Bertrand, On 11/09/2020 14:56, Bertrand Marquis wrote: On 11 Sep 2020, at 14:51, Julien Grall wrote: Hi Bertrand, On 11/09/2020 14:32, Bertrand Marquis wrote: On 11 Sep 2020, at 14:11, Jan Beulich wrote: All, the releases are about due, but will of course want to wait for the XSA

Re: [PATCH] xen/hypfs: fix writing of custom parameter

2020-09-11 Thread Jürgen Groß
On 11.09.20 16:02, Andrew Cooper wrote: On 11/09/2020 13:28, Jürgen Groß wrote: On 11.09.20 14:14, Andrew Cooper wrote: On 11/09/2020 06:30, Juergen Gross wrote: Today the maximum allowed data length for writing a hypfs node is tested in the generic hypfs_write() function. For custom runtime

Re: preparations for 4.13.2 and 4.12.4

2020-09-11 Thread Bertrand Marquis
> On 11 Sep 2020, at 14:51, Julien Grall wrote: > > Hi Bertrand, > > On 11/09/2020 14:32, Bertrand Marquis wrote: >>> On 11 Sep 2020, at 14:11, Jan Beulich wrote: >>> >>> All, >>> >>> the releases are about due, but will of course want to wait for the >>> XSA fixes going public on the

Re: preparations for 4.13.2 and 4.12.4

2020-09-11 Thread Julien Grall
Hi Bertrand, On 11/09/2020 14:32, Bertrand Marquis wrote: On 11 Sep 2020, at 14:11, Jan Beulich wrote: All, the releases are about due, but will of course want to wait for the XSA fixes going public on the 22nd. Please point out backports you find missing from the respective staging

Re: preparations for 4.13.2 and 4.12.4

2020-09-11 Thread Jan Beulich
On 11.09.2020 15:32, Bertrand Marquis wrote: > > >> On 11 Sep 2020, at 14:11, Jan Beulich wrote: >> >> All, >> >> the releases are about due, but will of course want to wait for the >> XSA fixes going public on the 22nd. Please point out backports >> you find missing from the respective staging

Re: [PATCH 3/8] Convert code-of-conduct to rst

2020-09-11 Thread Jürgen Groß
On 11.09.20 14:40, George Dunlap wrote: Underline section titles. Convert links to RST-style links. NB that the Communication Guide link won't work ATM; this will be fixed when we convert that document to RST. Adjust the formatting for the list so that it's converted properly. A couple of

Re: preparations for 4.13.2 and 4.12.4

2020-09-11 Thread Bertrand Marquis
> On 11 Sep 2020, at 14:11, Jan Beulich wrote: > > All, > > the releases are about due, but will of course want to wait for the > XSA fixes going public on the 22nd. Please point out backports > you find missing from the respective staging branches, but which > you consider relevant. (Ian,

Re: [PATCH 4/5] x86/pv: Optimise to the segment context switching paths

2020-09-11 Thread Jan Beulich
On 11.09.2020 14:53, Andrew Cooper wrote: > On 11/09/2020 10:49, Jan Beulich wrote: >> On 09.09.2020 11:59, Andrew Cooper wrote: >>> Save the segment selectors with explicit asm, rather than with read_sreg(). >>> This permits the use of MOV's m16 encoding, which avoids indirecting the >>> selector

[xen-unstable test] 154090: regressions - FAIL

2020-09-11 Thread osstest service owner
flight 154090 xen-unstable real [real] http://logs.test-lab.xenproject.org/osstest/logs/154090/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-amd64-pygrub 11 guest-start fail REGR. vs. 154016

[PATCH] tools/libs/stat: fix broken build

2020-09-11 Thread Juergen Gross
Making getBridge() static triggered a build error with some gcc versions: error: 'strncpy' output may be truncated copying 15 bytes from a string of length 255 [-Werror=stringop-truncation] Fix that by printing a sane error message and bailing out in case the name of a bridge is too long.

preparations for 4.13.2 and 4.12.4

2020-09-11 Thread Jan Beulich
All, the releases are about due, but will of course want to wait for the XSA fixes going public on the 22nd. Please point out backports you find missing from the respective staging branches, but which you consider relevant. (Ian, Julien, Stefano: I notice there once again haven't been any tools

Re: [PATCH] x86/PV: make post-migration page state consistent

2020-09-11 Thread Jan Beulich
On 11.09.2020 13:55, Andrew Cooper wrote: > On 11/09/2020 11:34, Jan Beulich wrote: >> When a page table page gets de-validated, its type reference count drops >> to zero (and PGT_validated gets cleared), but its type remains intact. >> XEN_DOMCTL_getpageframeinfo3, therefore, so far reported

Re: [PATCH v2] Arm64: force gcc 10+ to always inline generic atomics helpers

2020-09-11 Thread Bertrand Marquis
> On 11 Sep 2020, at 11:45, Jan Beulich wrote: > > Recent versions of gcc (at least 10.x) will not inline generic atomics > helpers by default. Instead they will expect the software to either link > with libatomic.so or implement the helpers, which would result in > > undefined reference to

Re: [PATCH v2] Arm64: force gcc 10+ to always inline generic atomics helpers

2020-09-11 Thread Julien Grall
Hi Jan, On 11/09/2020 11:45, Jan Beulich wrote: Recent versions of gcc (at least 10.x) will not inline generic atomics helpers by default. Instead they will expect the software to either link with libatomic.so or implement the helpers, which would result in undefined reference to

Re: [PATCH] xen/hypfs: fix writing of custom parameter

2020-09-11 Thread Jürgen Groß
On 11.09.20 14:14, Andrew Cooper wrote: On 11/09/2020 06:30, Juergen Gross wrote: Today the maximum allowed data length for writing a hypfs node is tested in the generic hypfs_write() function. For custom runtime parameters this might be wrong, as the maximum allowed size is derived from the

Re: [PATCH] x86/mm: do not mark IO regions as Xen heap

2020-09-11 Thread Jan Beulich
On 10.09.2020 16:41, Jan Beulich wrote: > On 10.09.2020 15:35, Roger Pau Monne wrote: >> arch_init_memory will treat all the gaps on the physical memory map >> between RAM regions as MMIO and use share_xen_page_with_guest in order >> to assign them to dom_io. This has the side effect of setting

[PATCH v2] Arm64: force gcc 10+ to always inline generic atomics helpers

2020-09-11 Thread Jan Beulich
Recent versions of gcc (at least 10.x) will not inline generic atomics helpers by default. Instead they will expect the software to either link with libatomic.so or implement the helpers, which would result in undefined reference to `__aarch64_ldadd4_acq_rel' for us (not having any local

Ping: [PATCH v2 2/2] EFI: free unused boot mem in at least some cases

2020-09-11 Thread Jan Beulich
On 24.08.2020 14:08, Jan Beulich wrote: > Address at least the primary reason why 52bba67f8b87 ("efi/boot: Don't > free ebmalloc area at all") was put in place: Make xen_in_range() aware > of the freed range. This is in particular relevant for EFI-enabled > builds not actually running on EFI, as

[xen-unstable bisection] complete test-amd64-amd64-pygrub

2020-09-11 Thread osstest service owner
branch xen-unstable xenbranch xen-unstable job test-amd64-amd64-pygrub testid debian-di-install Tree: linux git://xenbits.xen.org/linux-pvops.git Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git Tree: qemuu

[PATCH v4 8/8] hv_balloon: try to merge system ram resources

2020-09-11 Thread David Hildenbrand
Let's try to merge system ram resources we add, to minimize the number of resources in /proc/iomem. We don't care about the boundaries of individual chunks we added. Reviewed-by: Wei Liu Cc: Andrew Morton Cc: Michal Hocko Cc: "K. Y. Srinivasan" Cc: Haiyang Zhang Cc: Stephen Hemminger Cc:

[PATCH v4 6/8] virtio-mem: try to merge system ram resources

2020-09-11 Thread David Hildenbrand
virtio-mem adds memory in memory block granularity, to be able to remove it in the same granularity again later, and to grow slowly on demand. This, however, results in quite a lot of resources when adding a lot of memory. Resources are effectively stored in a list-based tree. Having a lot of

[PATCH v4 5/8] mm/memory_hotplug: MEMHP_MERGE_RESOURCE to specify merging of System RAM resources

2020-09-11 Thread David Hildenbrand
Some add_memory*() users add memory in small, contiguous memory blocks. Examples include virtio-mem, hyper-v balloon, and the XEN balloon. This can quickly result in a lot of memory resources, whereby the actual resource boundaries are not of interest (e.g., it might be relevant for DIMMs,

[PATCH v4 3/8] mm/memory_hotplug: guard more declarations by CONFIG_MEMORY_HOTPLUG

2020-09-11 Thread David Hildenbrand
We soon want to pass flags via a new type to add_memory() and friends. That revealed that we currently don't guard some declarations by CONFIG_MEMORY_HOTPLUG. While some definitions could be moved to different places, let's keep it minimal for now and use CONFIG_MEMORY_HOTPLUG for all functions

[PATCH v4 4/8] mm/memory_hotplug: prepare passing flags to add_memory() and friends

2020-09-11 Thread David Hildenbrand
We soon want to pass flags, e.g., to mark added System RAM resources. mergeable. Prepare for that. This patch is based on a similar patch by Oscar Salvador: https://lkml.kernel.org/r/20190625075227.15193-3-osalva...@suse.de Acked-by: Wei Liu Reviewed-by: Juergen Gross # Xen related part

[PATCH v4 2/8] kernel/resource: move and rename IORESOURCE_MEM_DRIVER_MANAGED

2020-09-11 Thread David Hildenbrand
IORESOURCE_MEM_DRIVER_MANAGED currently uses an unused PnP bit, which is always set to 0 by hardware. This is far from beautiful (and confusing), and the bit only applies to SYSRAM. So let's move it out of the bus-specific (PnP) defined bits. We'll add another SYSRAM specific bit soon. If we ever

[PATCH v4 7/8] xen/balloon: try to merge system ram resources

2020-09-11 Thread David Hildenbrand
Let's try to merge system ram resources we add, to minimize the number of resources in /proc/iomem. We don't care about the boundaries of individual chunks we added. Reviewed-by: Juergen Gross Cc: Andrew Morton Cc: Michal Hocko Cc: Boris Ostrovsky Cc: Juergen Gross Cc: Stefano Stabellini

[PATCH v4 1/8] kernel/resource: make release_mem_region_adjustable() never fail

2020-09-11 Thread David Hildenbrand
Let's make sure splitting a resource on memory hotunplug will never fail. This will become more relevant once we merge selected System RAM resources - then, we'll trigger that case more often on memory hotunplug. In general, this function is already unlikely to fail. When we remove memory, we

[PATCH v4 0/8] selective merging of system ram resources

2020-09-11 Thread David Hildenbrand
Some add_memory*() users add memory in small, contiguous memory blocks. Examples include virtio-mem, hyper-v balloon, and the XEN balloon. This can quickly result in a lot of memory resources, whereby the actual resource boundaries are not of interest (e.g., it might be relevant for DIMMs,

[PATCH] x86/PV: fold exit paths of ptwr_do_page_fault()

2020-09-11 Thread Jan Beulich
One less aspect to keep an eye on for things to stay in sync. Signed-off-by: Jan Beulich --- FAOD I did consider (and even try) reducing the scope of ptwr_ctxt at the same occasion - this results in worse code with gcc 10 at least, as the compiler then indeed defers populating of the struct, and

[PATCH] x86/PV: make post-migration page state consistent

2020-09-11 Thread Jan Beulich
When a page table page gets de-validated, its type reference count drops to zero (and PGT_validated gets cleared), but its type remains intact. XEN_DOMCTL_getpageframeinfo3, therefore, so far reported prior usage for such pages. An intermediate write to such a page via e.g. MMU_NORMAL_PT_UPDATE,

Re: [PATCH V1 09/16] arm/ioreq: Introduce arch specific bits for IOREQ/DM features

2020-09-11 Thread Oleksandr
Hello all. /* diff --git a/xen/include/asm-arm/hvm/ioreq.h b/xen/include/asm-arm/hvm/ioreq.h new file mode 100644 index 000..1c34df0 --- /dev/null +++ b/xen/include/asm-arm/hvm/ioreq.h @@ -0,0 +1,108 @@ +/* + * hvm.h: Hardware virtual machine assist interface definitions. + * + *

Re: [PATCH 5/5] x86/pv: Simplify emulation for the 64bit base MSRs

2020-09-11 Thread Jan Beulich
On 09.09.2020 11:59, Andrew Cooper wrote: > is_pv_32bit_domain() is an expensive predicate, but isn't used for speculative > safety in this case. Swap to checking the Long Mode bit in the CPUID policy, > which is the architecturally correct behaviour. > > is_canonical_address() isn't a trivial

Re: [PATCH 4/5] x86/pv: Optimise to the segment context switching paths

2020-09-11 Thread Jan Beulich
On 09.09.2020 11:59, Andrew Cooper wrote: > Save the segment selectors with explicit asm, rather than with read_sreg(). > This permits the use of MOV's m16 encoding, which avoids indirecting the > selector value through a register. Instead of this, how about making read_sreg() look like #define

Re: [PATCH] xen/hypfs: fix writing of custom parameter

2020-09-11 Thread Jan Beulich
On 11.09.2020 07:30, Juergen Gross wrote: > Today the maximum allowed data length for writing a hypfs node is > tested in the generic hypfs_write() function. For custom runtime > parameters this might be wrong, as the maximum allowed size is derived > from the buffer holding the current setting,

[PATCH v8 1/8] x86/iommu: convert VT-d code to use new page table allocator

2020-09-11 Thread Paul Durrant
From: Paul Durrant This patch converts the VT-d code to use the new IOMMU page table allocator function. This allows all the free-ing code to be removed (since it is now handled by the general x86 code) which reduces TLB and cache thrashing as well as shortening the code. The scope of the

[PATCH v8 6/8] common/grant_table: batch flush I/O TLB

2020-09-11 Thread Paul Durrant
From: Paul Durrant This patch avoids calling iommu_iotlb_flush() for each individual GNTTABOP and instead calls iommu_iotlb_flush_all() at the end of a batch. This should mean non-singleton map/unmap operations perform better. NOTE: A batch is the number of operations done before a pre-emption

[PATCH v8 2/8] iommu: remove unused iommu_ops method and tasklet

2020-09-11 Thread Paul Durrant
From: Paul Durrant The VT-d and AMD IOMMU implementations both use the general x86 IOMMU page table allocator and ARM always shares page tables with CPU. Hence there is no need to retain the free_page_table() method or the tasklet which invokes it. Signed-off-by: Paul Durrant Reviewed-by: Jan

[PATCH v8 0/8] IOMMU cleanup

2020-09-11 Thread Paul Durrant
From: Paul Durrant Paul Durrant (8): x86/iommu: convert VT-d code to use new page table allocator iommu: remove unused iommu_ops method and tasklet iommu: flush I/O TLB if iommu_map() or iommu_unmap() fail iommu: make map and unmap take a page count, similar to flush remove remaining

[PATCH v8 8/8] iommu: stop calling IOMMU page tables 'p2m tables'

2020-09-11 Thread Paul Durrant
From: Paul Durrant It's confusing and not consistent with the terminology introduced with 'dfn_t'. Just call them IOMMU page tables. Also remove a pointless check of the 'acpi_drhd_units' list in vtd_dump_page_table_level(). If the list is empty then IOMMU mappings would not have been enabled

[PATCH v8 3/8] iommu: flush I/O TLB if iommu_map() or iommu_unmap() fail

2020-09-11 Thread Paul Durrant
From: Paul Durrant This patch adds a full I/O TLB flush to the error paths of iommu_map() and iommu_unmap(). Without this change callers need constructs such as: rc = iommu_map/unmap(...) err = iommu_flush(...) if ( !rc ) rc = err; With this change, it can be simplified to: rc =

Re: [PATCH v3 3/7] mm/memory_hotplug: prepare passing flags to add_memory() and friends

2020-09-11 Thread David Hildenbrand
On 11.09.20 04:21, kernel test robot wrote: > Hi David, > > I love your patch! Yet something to improve: > > [auto build test ERROR on next-20200909] > [cannot apply to mmotm/master hnaz-linux-mm/master xen-tip/linux-next > powerpc/next linus/master v5.9-rc4 v5.9-rc3 v5.9-rc2 v5.9-rc4] > [If

[PATCH v8 4/8] iommu: make map and unmap take a page count, similar to flush

2020-09-11 Thread Paul Durrant
From: Paul Durrant At the moment iommu_map() and iommu_unmap() take a page order rather than a count, whereas iommu_iotlb_flush() takes a page count rather than an order. This patch makes them consistent with each other, opting for a page count since CPU page orders are not necessarily the same

[PATCH v8 5/8] remove remaining uses of iommu_legacy_map/unmap

2020-09-11 Thread Paul Durrant
From: Paul Durrant The 'legacy' functions do implicit flushing so amend the callers to do the appropriate flushing. Unfortunately, because of the structure of the P2M code, we cannot remove the per-CPU 'iommu_dont_flush_iotlb' global and the optimization it facilitates. It is now checked

[PATCH v8 7/8] iommu: remove the share_p2m operation

2020-09-11 Thread Paul Durrant
From: Paul Durrant Sharing of HAP tables is now VT-d specific so the operation is never defined for AMD IOMMU any more. There's also no need to pro-actively set vtd.pgd_maddr when using shared EPT as it is straightforward to simply define a helper function to return the appropriate value in the

Re: [PATCH v6 4/8] iommu: make map and unmap take a page count, similar to flush

2020-09-11 Thread Oleksandr
On 11.09.20 10:59, Paul Durrant wrote: Hi De-htmling... - From: Oleksandr Tyshchenko Sent: 10 September 2020 19:20 To: Durrant, Paul Cc: Bertrand Marquis ; Paul Durrant ; open list:X86 ; Jan Beulich ; Andrew Cooper ; Wei Liu ; Roger Pau Monné ; George Dunlap ; Ian Jackson ; Julien

RE: [PATCH v6 4/8] iommu: make map and unmap take a page count, similar to flush

2020-09-11 Thread Paul Durrant
De-htmling... - From: Oleksandr Tyshchenko Sent: 10 September 2020 19:20 To: Durrant, Paul Cc: Bertrand Marquis ; Paul Durrant ; open list:X86 ; Jan Beulich ; Andrew Cooper ; Wei Liu ; Roger Pau Monné ; George Dunlap ; Ian Jackson ; Julien Grall ; Stefano Stabellini ; Jun Nakajima ;

[libvirt test] 154097: regressions - FAIL

2020-09-11 Thread osstest service owner
flight 154097 libvirt real [real] http://logs.test-lab.xenproject.org/osstest/logs/154097/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: build-i386-libvirt6 libvirt-buildfail REGR. vs. 151777 build-amd64-libvirt

[linux-linus test] 154078: regressions - FAIL

2020-09-11 Thread osstest service owner
flight 154078 linux-linus real [real] http://logs.test-lab.xenproject.org/osstest/logs/154078/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-boot fail REGR. vs. 152332

Re: [PATCH 3/5] x86/pv: Optimise prefetching in svm_load_segs()

2020-09-11 Thread Jan Beulich
On 10.09.2020 22:30, Andrew Cooper wrote: > On 10/09/2020 15:57, Jan Beulich wrote: >> On 09.09.2020 11:59, Andrew Cooper wrote: >>> Split into two functions. Passing a load of zeros in results in somewhat >>> poor >>> register scheduling in __context_switch(). >> I'm afraid I don't understand

Re: [PATCH 01/11 v2] gitignore: Move ignores from global to subdirectories

2020-09-11 Thread Jan Beulich
On 10.09.2020 20:22, Elliott Mitchell wrote: > On Thu, Sep 10, 2020 at 11:13:26AM +0200, Jan Beulich wrote: >> On 27.08.2020 21:09, Elliott Mitchell wrote: >>> --- a/tools/misc/.gitignore >>> +++ b/tools/misc/.gitignore >>> @@ -1 +1,22 @@ >>> -xen-ucode >>> +/cpuperf/cpuperf-perfcntr >>>

Re: [PATCH] x86/mm: do not mark IO regions as Xen heap

2020-09-11 Thread Jan Beulich
On 10.09.2020 19:27, Roger Pau Monné wrote: > On Thu, Sep 10, 2020 at 04:41:41PM +0200, Jan Beulich wrote: >> On 10.09.2020 15:35, Roger Pau Monne wrote: >>> --- a/xen/arch/x86/mm.c >>> +++ b/xen/arch/x86/mm.c >>> @@ -271,6 +271,18 @@ static l4_pgentry_t __read_mostly split_l4e; >>> #define