d by calling vm_raw_write_end() by the callee once the ptes have
been moved.
Signed-off-by: Laurent Dufour
---
include/linux/mm.h | 24 +++-
mm/mmap.c | 53 +
mm/mremap.c| 13 +
3 files changed, 73 inser
prevent write to be split
and intermediate values to be pushed to other CPUs.
Signed-off-by: Laurent Dufour
---
fs/proc/task_mmu.c | 5 -
fs/userfaultfd.c | 17 +
mm/khugepaged.c| 3 +++
mm/madvise.c | 6 +-
mm/mempolicy.c | 51
ve sequence initialisation INIT_VMA()]
Signed-off-by: Laurent Dufour
---
include/linux/mm.h | 44
include/linux/mm_types.h | 3 +++
mm/memory.c | 2 ++
mm/mmap.c| 31 +++
4 files changed, 80
cated inline function.
Signed-off-by: Laurent Dufour
---
fs/exec.c | 2 +-
include/linux/mm.h | 5 +
kernel/fork.c | 2 +-
mm/mmap.c | 10 +-
mm/nommu.c | 2 +-
5 files changed, 13 insertions(+), 8 deletions(-)
diff --git a/fs/exec.c b/fs/exec.c
ch is then trapped in cow_user_page().
If VM_FAULT_RETRY is returned, it is passed up to the callers to retry the
page fault while holding the mmap_sem.
Acked-by: David Rientjes
Signed-off-by: Laurent Dufour
---
include/linux/mm.h | 1 +
mm/memory.c| 39 --
When handling page fault without holding the mmap_sem the fetch of the
pte lock pointer and the locking will have to be done while ensuring
that the VMA is not touched in our back.
So move the fetch and locking operations in a dedicated function.
Signed-off-by: Laurent Dufour
---
mm/memory.c
n the file]
[move the define of FAULT_FLAG_SPECULATIVE later in the series]
[review error path in do_swap_page(), do_anonymous_page() and
wp_page_copy()]
Signed-off-by: Laurent Dufour
---
mm/memory.c | 87 -
1 file changed, 58 inserti
t for book3e_hugetlb_preload()
called by update_mmu_cache()
Cc: Michael Ellerman
Signed-off-by: Laurent Dufour
---
arch/powerpc/Kconfig | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index c32a181a7cbb..21ef887da7a3 100644
--- a/arch/powerpc/Kconfig
+++ b
Set ARCH_SUPPORTS_SPECULATIVE_PAGE_FAULT which turns on the
Speculative Page Fault handler when building for 64bit.
Cc: Thomas Gleixner
Signed-off-by: Laurent Dufour
---
arch/x86/Kconfig | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index d8983df5a2bc
This configuration variable will be used to build the code needed to
handle speculative page fault.
By default it is turned off, and activated depending on architecture
support, SMP and MMU.
Suggested-by: Thomas Gleixner
Suggested-by: David Rientjes
Signed-off-by: Laurent Dufour
---
mm
ew_anon_rmap() is called during the spf path and protect against
anon_vma pointer's update.
- In patch 13-16, add performance events to report number of successful
and failed speculative events.
[1]
http://linux-kernel.2935.n7.nabble.com/RFC-PATCH-0-6-Another-go-at-speculative
On 14/03/2018 14:11, Michal Hocko wrote:
> On Tue 13-03-18 18:59:30, Laurent Dufour wrote:
>> Changes since v8:
>> - Don't check PMD when locking the pte when THP is disabled
>>Thanks to Daniel Jordan for reporting this.
>> - Rebase on 4.16
>
> Is this
On 10/04/2018 08:47, David Rientjes wrote:
> On Mon, 26 Mar 2018, Andi Kleen wrote:
>
>>> Aside: should there be a new spec_flt field for struct task_struct that
>>> complements maj_flt and min_flt and be exported through /proc/pid/stat?
>>
>> No. task_struct is already too bloated. If you need p
Remove the additional define HAVE_PTE_SPECIAL and rely directly on
CONFIG_ARCH_HAS_PTE_SPECIAL.
There is no functional change introduced by this patch
Signed-off-by: Laurent Dufour
---
mm/memory.c | 15 ++-
1 file changed, 6 insertions(+), 9 deletions(-)
diff --git a/mm/memory.c b
On 11/04/2018 11:09, Christophe LEROY wrote:
>
>
> Le 11/04/2018 à 11:03, Laurent Dufour a écrit :
>>
>>
>> On 11/04/2018 10:58, Christophe LEROY wrote:
>>>
>>>
>>> Le 11/04/2018 à 10:03, Laurent Dufour a écrit :
>>>>
On 11/04/2018 10:58, Christophe LEROY wrote:
>
>
> Le 11/04/2018 à 10:03, Laurent Dufour a écrit :
>> Remove the additional define HAVE_PTE_SPECIAL and rely directly on
>> CONFIG_ARCH_HAS_PTE_SPECIAL.
>>
>> There is no functional change introduced by this p
On 11/04/2018 10:33, Michal Hocko wrote:
> On Wed 11-04-18 10:03:36, Laurent Dufour wrote:
>> @@ -881,7 +876,8 @@ struct page *_vm_normal_page(struct vm_area_struct *vma,
>> unsigned long addr,
>>
>> if (is_zero_pfn(pfn))
>> return NULL;
E_SPECIAL if SPARC64
There is no functional change introduced by this patch.
Suggested-by: Jerome Glisse
Reviewed-by: Jerome Glisse
Acked-by: David Rientjes
Signed-off-by: Laurent Dufour
---
Documentation/features/vm/pte_special/arch-support.txt | 2 +-
arch
Remove the additional define HAVE_PTE_SPECIAL and rely directly on
CONFIG_ARCH_HAS_PTE_SPECIAL.
There is no functional change introduced by this patch
Signed-off-by: Laurent Dufour
---
mm/memory.c | 19 ---
1 file changed, 8 insertions(+), 11 deletions(-)
diff --git a/mm
emove __HAVE_ARCH_PTE_SPECIAL in arch/riscv/include/asm/pgtable-bits.h
* use IS_ENABLED() instead of #ifdef blocks in patch 2
Laurent Dufour (2):
mm: introduce ARCH_HAS_PTE_SPECIAL
mm: remove odd HAVE_PTE_SPECIAL
.../features/vm/pte_special/arch-support.txt | 2 +-
arch/arc/Kc
On 10/04/2018 17:58, Robin Murphy wrote:
> On 10/04/18 16:25, Laurent Dufour wrote:
>> Remove the additional define HAVE_PTE_SPECIAL and rely directly on
>> CONFIG_ARCH_HAS_PTE_SPECIAL.
>>
>> There is no functional change introduced by this patch
>>
On 10/04/2018 18:09, Matthew Wilcox wrote:
> On Tue, Apr 10, 2018 at 05:25:50PM +0200, Laurent Dufour wrote:
>> arch/powerpc/include/asm/pte-common.h | 3 ---
>> arch/riscv/Kconfig | 1 +
>&g
On 03/04/2018 01:57, David Rientjes wrote:
> On Tue, 13 Mar 2018, Laurent Dufour wrote:
>
>> When dealing with speculative page fault handler, we may race with VMA
>> being split or merged. In this case the vma->vm_start and vm->vm_end
>> fields may not match
On 03/04/2018 02:11, David Rientjes wrote:
> On Tue, 13 Mar 2018, Laurent Dufour wrote:
>
>> This change is inspired by the Peter's proposal patch [1] which was
>> protecting the VMA using SRCU. Unfortunately, SRCU is not scaling well in
>> that particular case
E_SPECIAL if SPARC64
There is no functional change introduced by this patch.
Suggested-by: Jerome Glisse
Reviewed-by: Jerome Glisse
Signed-off-by: Laurent Dufour
---
Documentation/features/vm/pte_special/arch-support.txt | 2 +-
arch/arc/Kconfig | 1 +
arch
Remove the additional define HAVE_PTE_SPECIAL and rely directly on
CONFIG_ARCH_HAS_PTE_SPECIAL.
There is no functional change introduced by this patch
Signed-off-by: Laurent Dufour
---
mm/memory.c | 23 ++-
1 file changed, 10 insertions(+), 13 deletions(-)
diff --git a/mm
onfig files,
setting it automatically when architectures was already setting it in
header file.
The second patch is removing the odd define HAVE_PTE_SPECIAL which is a
duplicate of CONFIG_ARCH_HAS_PTE_SPECIAL.
There is no functional change introduced by this series.
Laurent Dufour (2):
mm: intr
On 09/04/2018 22:08, David Rientjes wrote:
> On Mon, 9 Apr 2018, Christoph Hellwig wrote:
>
>>> -#ifdef __HAVE_ARCH_PTE_SPECIAL
>>> +#ifdef CONFIG_ARCH_HAS_PTE_SPECIAL
>>> # define HAVE_PTE_SPECIAL 1
>>> #else
>>> # define HAVE_PTE_SPECIAL 0
>>
>> I'd say kill this odd indirection and just use
On 09/04/2018 18:03, Vineet Gupta wrote:
> On 04/09/2018 06:57 AM, Laurent Dufour wrote:
>> The per architecture __HAVE_ARCH_PTE_SPECIAL is defined statically in the
>> per architecture header files. This doesn't allow to make other
>> configuration dependent on it.
It is now replaced by Kconfig variable CONFIG_ARCH_HAS_PTE_SPECIAL.
Signed-off-by: Laurent Dufour
---
arch/arc/include/asm/pgtable.h | 2 --
arch/arm/include/asm/pgtable-3level.h| 1 -
arch/arm64/include/asm/pgtable.h | 2 --
arch/powerpc/include/asm/book3s/64
E_SPECIAL if SPARC64
Suggested-by: Jerome Glisse
Signed-off-by: Laurent Dufour
---
arch/arc/Kconfig | 1 +
arch/arm/Kconfig | 1 +
arch/arm64/Kconfig | 1 +
arch/powerpc/Kconfig | 1 +
arch/riscv/Kconfig | 1 +
arch/s390/Kconfig| 1 +
arch/sh/Kconfig | 1 +
arch/sparc/Kconfig
Replace __HAVE_ARCH_PTE_SPECIAL by the new configuration variable
CONFIG_ARCH_HAS_PTE_SPECIAL.
Signed-off-by: Laurent Dufour
---
Documentation/features/vm/pte_special/arch-support.txt | 2 +-
include/linux/pfn_t.h | 4 ++--
mm/gup.c
s was already setting it in
header file.
There is no functional change introduced by this series.
Laurent Dufour (3):
mm: introduce ARCH_HAS_PTE_SPECIAL
mm: replace __HAVE_ARCH_PTE_SPECIAL
mm: remove __HAVE_ARCH_PTE_SPECIAL
Documentation/features/vm/pte_special/arch-support.txt | 2 +-
arc
On 03/04/2018 02:11, David Rientjes wrote:
> On Tue, 13 Mar 2018, Laurent Dufour wrote:
>
>> This change is inspired by the Peter's proposal patch [1] which was
>> protecting the VMA using SRCU. Unfortunately, SRCU is not scaling well in
>> that particular case
On 04/04/2018 23:53, David Rientjes wrote:
> On Wed, 4 Apr 2018, Laurent Dufour wrote:
>
>>> I also think the following is needed:
>>>
>>> diff --git a/fs/exec.c b/fs/exec.c
>>> --- a/fs/exec.c
>>> +++ b/fs/exec.c
>>> @@ -312,6 +3
On 04/04/2018 23:59, Jerome Glisse wrote:
> On Wed, Apr 04, 2018 at 06:26:44PM +0200, Laurent Dufour wrote:
>>
>>
>> On 03/04/2018 21:39, Jerome Glisse wrote:
>>> On Tue, Mar 13, 2018 at 06:59:45PM +0100, Laurent Dufour wrote:
>>>> When dealing with th
On 03/04/2018 21:39, Jerome Glisse wrote:
> On Tue, Mar 13, 2018 at 06:59:45PM +0100, Laurent Dufour wrote:
>> When dealing with the speculative fault path we should use the VMA's field
>> cached value stored in the vm_fault structure.
>>
>> Currently vm_normal_pa
On 03/04/2018 01:18, David Rientjes wrote:
> On Tue, 13 Mar 2018, Laurent Dufour wrote:
>
>> diff --git a/include/linux/mm.h b/include/linux/mm.h
>> index a84ddc218bbd..73b8b99f482b 100644
>> --- a/include/linux/mm.h
>> +++ b/include/linux/mm.h
>> @@ -1
On 03/04/2018 01:12, David Rientjes wrote:
> On Tue, 13 Mar 2018, Laurent Dufour wrote:
>
>> diff --git a/include/linux/mm.h b/include/linux/mm.h
>> index dfa81a638b7c..a84ddc218bbd 100644
>> --- a/include/linux/mm.h
>> +++ b/include/linux/mm.h
>> @@ -684,1
On 03/04/2018 00:24, David Rientjes wrote:
> On Tue, 13 Mar 2018, Laurent Dufour wrote:
>
>> diff --git a/include/linux/mm.h b/include/linux/mm.h
>> index ef6ef0627090..dfa81a638b7c 100644
>> --- a/include/linux/mm.h
>> +++ b/include/linux/mm.h
>>
On 04/04/2018 03:03, David Rientjes wrote:
> On Tue, 3 Apr 2018, David Rientjes wrote:
>
> I found the root cause of this lockdep warning.
>
> In mmap_region(), unmap_region() may be called while vma_link() has not
> been
> called. This happens during the error path if call_
On 04/04/2018 02:48, David Rientjes wrote:
> On Wed, 28 Mar 2018, Laurent Dufour wrote:
>
>> On 26/03/2018 00:10, David Rientjes wrote:
>>> On Wed, 21 Mar 2018, Laurent Dufour wrote:
>>>
>>>> I found the root cause of this lockdep warning.
>>
On 03/04/2018 21:10, Jerome Glisse wrote:
> On Tue, Mar 13, 2018 at 06:59:36PM +0100, Laurent Dufour wrote:
>> pte_unmap_same() is making the assumption that the page table are still
>> around because the mmap_sem is held.
>> This is no more the case when running a specu
On 03/04/2018 23:57, David Rientjes wrote:
> On Wed, 28 Mar 2018, Laurent Dufour wrote:
>
>>>> diff --git a/include/linux/mm.h b/include/linux/mm.h
>>>> index 4d02524a7998..2f3e98edc94a 100644
>>>> --- a/include/linux/mm.h
>>>> +++ b/inclu
On 28/03/2018 23:21, David Rientjes wrote:
> On Wed, 28 Mar 2018, Laurent Dufour wrote:
>
>>>> @@ -326,7 +336,10 @@ static unsigned long move_vma(struct vm_area_struct
>>>> *vma,
>>>>mremap_userfaultfd_prep(new_vma, uf);
>>>>
Hi Jerome,
Thanks for reviewing this series.
On 03/04/2018 22:37, Jerome Glisse wrote:
> On Tue, Mar 13, 2018 at 06:59:30PM +0100, Laurent Dufour wrote:
>> This is a port on kernel 4.16 of the work done by Peter Zijlstra to
>> handle page fault without holding the mm semaphore
On 03/04/2018 15:54, Frederic Barrat wrote:
> cxllib_handle_fault() is called by an external driver when it needs to
> have the host resolve page faults for a buffer. The buffer can cover
> several pages and VMAs. The function iterates over all the pages used
> by the buffer, based on the page size
driver will raise a new page
> fault when retrying.
>
> Fixes: 3ced8d730063 ("cxl: Export library to support IBM XSL")
> Cc: sta...@vger.kernel.org # 4.13+
> Signed-off-by: Frederic Barrat
FWIW,
Reviewed-by : Laurent Dufour
> ---
> drivers/misc/cxl/cxllib.c | 85
On 22/03/2018 02:21, Ganesh Mahendran wrote:
> Hi, Laurent
>
> 2018-03-14 1:59 GMT+08:00 Laurent Dufour :
>> This is a port on kernel 4.16 of the work done by Peter Zijlstra to
>> handle page fault without holding the mm semaphore [1].
>>
>> The idea is to try
On 29/03/2018 05:06, Ganesh Mahendran wrote:
> 2018-03-29 10:26 GMT+08:00 Ganesh Mahendran :
>> Hi, Laurent
>>
>> 2018-02-16 23:25 GMT+08:00 Laurent Dufour :
>>> When the speculative page fault handler is returning VM_RETRY, there is a
>>> chance that VMA fe
On 28/03/2018 00:12, David Rientjes wrote:
> On Tue, 13 Mar 2018, Laurent Dufour wrote:
>
>> diff --git a/include/linux/mm.h b/include/linux/mm.h
>> index 88042d843668..ef6ef0627090 100644
>> --- a/include/linux/mm.h
>> +++ b/include/linux/mm.h
&
On 27/03/2018 23:30, David Rientjes wrote:
> On Tue, 13 Mar 2018, Laurent Dufour wrote:
>
>> diff --git a/mm/mmap.c b/mm/mmap.c
>> index faf85699f1a1..5898255d0aeb 100644
>> --- a/mm/mmap.c
>> +++ b/mm/mmap.c
>> @@ -558,6 +558,10 @@ void __vma_
On 27/03/2018 23:57, David Rientjes wrote:
> On Tue, 13 Mar 2018, Laurent Dufour wrote:
>
>> diff --git a/mm/mmap.c b/mm/mmap.c
>> index 5898255d0aeb..d6533cb85213 100644
>> --- a/mm/mmap.c
>> +++ b/mm/mmap.c
>> @@ -847,17 +847,18 @@ int __vma_adjust(
On 27/03/2018 23:45, David Rientjes wrote:
> On Tue, 13 Mar 2018, Laurent Dufour wrote:
>
>> diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
>> index 65ae54659833..a2d9c87b7b0b 100644
>> --- a/fs/proc/task_mmu.c
>> +++ b/fs/proc/task_mmu.c
>> @
On 26/03/2018 00:10, David Rientjes wrote:
> On Wed, 21 Mar 2018, Laurent Dufour wrote:
>
>> I found the root cause of this lockdep warning.
>>
>> In mmap_region(), unmap_region() may be called while vma_link() has not been
>> called. This happens during the er
On 28/03/2018 12:16, David Rientjes wrote:
> On Wed, 28 Mar 2018, Laurent Dufour wrote:
>
>>>> This configuration variable will be used to build the code needed to
>>>> handle speculative page fault.
>>>>
>>>> By default it is turned off
On 28/03/2018 12:20, David Rientjes wrote:
> On Wed, 28 Mar 2018, Laurent Dufour wrote:
>
>>>> @@ -2913,7 +2921,8 @@ int do_swap_page(struct vm_fault *vmf)
>>>>int exclusive = 0;
>>>>int ret = 0;
>>>
>>> Initialization is
On 25/03/2018 23:50, David Rientjes wrote:
> On Tue, 13 Mar 2018, Laurent Dufour wrote:
>
>> From: Peter Zijlstra
>>
>> When speculating faults (without holding mmap_sem) we need to validate
>> that the vma against which we loaded pages is still valid when we'
On 27/03/2018 23:18, David Rientjes wrote:
> On Tue, 13 Mar 2018, Laurent Dufour wrote:
>
>> diff --git a/include/linux/mm.h b/include/linux/mm.h
>> index 2f3e98edc94a..b6432a261e63 100644
>> --- a/include/linux/mm.h
>> +++ b/include/linux/mm.h
>> @@ -
On 25/03/2018 23:50, David Rientjes wrote:
> On Tue, 13 Mar 2018, Laurent Dufour wrote:
>
>> When handling page fault without holding the mmap_sem the fetch of the
>> pte lock pointer and the locking will have to be done while ensuring
>> that the VMA is not touched in our
Hi David,
Thanks a lot for your deep review on this series.
On 25/03/2018 23:50, David Rientjes wrote:
> On Tue, 13 Mar 2018, Laurent Dufour wrote:
>
>> This configuration variable will be used to build the code needed to
>> handle speculative page fault.
>>
>> By
On 17/03/2018 08:51, kernel test robot wrote:
> FYI, we noticed the following commit (built with gcc-7):
>
> commit: b1f0502d04537ef55b0c296823affe332b100eb5 ("mm: VMA sequence count")
> url:
> https://github.com/0day-ci/linux/commits/Laurent-Dufour/Speculative-pa
On 16/03/2018 11:23, kernel test robot wrote:
> FYI, we noticed the following commit (built with gcc-7):
>
> commit: b33ddf50ebcc740b990dd2e0e8ff0b92c7acf58e ("mm: Protect mm_rb tree
> with a rwlock")
> url:
> https://github.com/0day-ci/linux/commits/Laurent-Du
On 14/03/2018 09:48, Peter Zijlstra wrote:
> On Tue, Mar 13, 2018 at 06:59:47PM +0100, Laurent Dufour wrote:
>> This change is inspired by the Peter's proposal patch [1] which was
>> protecting the VMA using SRCU. Unfortunately, SRCU is not scaling well in
>> that
On 14/03/2018 14:11, Michal Hocko wrote:
> On Tue 13-03-18 18:59:30, Laurent Dufour wrote:
>> Changes since v8:
>> - Don't check PMD when locking the pte when THP is disabled
>>Thanks to Daniel Jordan for reporting this.
>> - Rebase on 4.16
>
> Is this
This patch a set of new trace events to collect the speculative page fault
event failures.
Signed-off-by: Laurent Dufour
---
include/trace/events/pagefault.h | 87
mm/memory.c | 62 ++--
2 files changed, 136
prevent write to be split
and intermediate values to be pushed to other CPUs.
Signed-off-by: Laurent Dufour
---
fs/proc/task_mmu.c | 5 -
fs/userfaultfd.c | 17 +
mm/khugepaged.c| 3 +++
mm/madvise.c | 6 +-
mm/mempolicy.c | 51
multithreaded process as there is no
risk of contention on the mmap_sem otherwise.
Build on if CONFIG_SPECULATIVE_PAGE_FAULT is defined (currently for
BOOK3S_64 && SMP).
Signed-off-by: Laurent Dufour
---
arch/powerpc/mm/fault.c | 31 ++-
1 file changed, 30 in
path in case of retry]
Signed-off-by: Laurent Dufour
---
arch/x86/mm/fault.c | 38 +-
1 file changed, 37 insertions(+), 1 deletion(-)
diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
index e6af2b464c3d..a73cf227edd6 100644
--- a/arch/x86/mm/fault.c
+++ b
aries matched the passed address and release
the reference on the VMA so that it can be freed if needed.
In the case the VMA is freed, can_reuse_spf_vma() will have returned false
as the VMA is no more in the RB tree.
Signed-off-by: Laurent Dufour
---
include/linux/mm.h | 5 +-
mm/memory.c
Add support for the new speculative faults event.
Signed-off-by: Laurent Dufour
---
tools/include/uapi/linux/perf_event.h | 1 +
tools/perf/util/evsel.c | 1 +
tools/perf/util/parse-events.c| 4
tools/perf/util/parse-events.l| 1 +
tools/perf/util/python.c
Add a new software event to count succeeded speculative page faults.
Signed-off-by: Laurent Dufour
---
include/uapi/linux/perf_event.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/include/uapi/linux/perf_event.h b/include/uapi/linux/perf_event.h
index 6f873503552d..a6ddab9edeec 100644
Check PMD against concurrent collapsing operation]
[Try spin lock the pte during the speculative path to avoid deadlock with
other CPU's invalidating the TLB and requiring this CPU to catch the
inter processor's interrupt]
Signed-off-by: Laurent Dufour
---
include/linux/hugetlb_inline.h
exiting an
in progress speculative page fault.
[1] https://patchwork.kernel.org/patch/5108281/
Cc: Peter Zijlstra (Intel)
Cc: Matthew Wilcox
Signed-off-by: Laurent Dufour
---
include/linux/mm_types.h | 4 ++
kernel/fork.c| 3 ++
mm/init-mm.c | 3 ++
mm/internal.h
_vma is still valid as explained
above.
Signed-off-by: Laurent Dufour
---
include/linux/rmap.h | 12 ++--
mm/memory.c | 8
mm/rmap.c| 5 ++---
3 files changed, 16 insertions(+), 9 deletions(-)
diff --git a/include/linux/rmap.h b/include/linux/rmap.h
index 988d
flags value as parameter.
Note: The speculative path is turned on for architecture providing support
for special PTE flag. So only the first block of vm_normal_page is used
during the speculative path.
Signed-off-by: Laurent Dufour
---
include/linux/mm.h | 7 +--
mm/memory.c
pointer.
Signed-off-by: Laurent Dufour
---
include/linux/swap.h | 10 --
mm/memory.c | 8
mm/swap.c| 6 +++---
3 files changed, 15 insertions(+), 9 deletions(-)
diff --git a/include/linux/swap.h b/include/linux/swap.h
index 1985940af479..a7dc37e0e405 100644
() service which can be called by
passing the value of the vm_flags field.
There is no change functional changes expected for the other callers of
maybe_mkwrite().
Signed-off-by: Laurent Dufour
---
include/linux/mm.h | 9 +++--
mm/memory.c| 6 +++---
2 files changed, 10 insertions(+), 5
hanges.
This patch also set the fields in hugetlb_no_page() and
__collapse_huge_page_swapin even if it is not need for the callee.
Signed-off-by: Laurent Dufour
---
include/linux/mm.h | 6 ++
mm/hugetlb.c | 2 ++
mm/khugepaged.c| 2 ++
mm/memory.c
migrate_misplaced_page() is only called during the page fault handling so
it's better to pass the pointer to the struct vm_fault instead of the vma.
This way during the speculative page fault path the saved vma->vm_flags
could be used.
Signed-off-by: Laurent Dufour
---
include/linux/m
or the vma sequence counter which is
updated in unmap_page_range() before locking the pte, and then in
free_pgtables() so when locking the pte the change will be detected.
Signed-off-by: Laurent Dufour
---
mm/memory.c | 4
1 file changed, 4 insertions(+)
diff --git a/mm/memory.c
d by calling vm_raw_write_end() by the callee once the ptes have
been moved.
Signed-off-by: Laurent Dufour
---
include/linux/mm.h | 16
mm/mmap.c | 47 ---
mm/mremap.c| 13 +
3 files changed, 61 insertions(+),
ch is then trapped in cow_user_page().
If VM_FAULT_RETRY is returned, it is passed up to the callers to retry the
page fault while holding the mmap_sem.
Signed-off-by: Laurent Dufour
---
include/linux/mm.h | 1 +
mm/memory.c| 29 +++--
2 files changed, 20 inserti
ds on CONFIG_SPECULATIVE_PAGE_FAULT]
[Introduce vm_write_* inline function depending on
CONFIG_SPECULATIVE_PAGE_FAULT]
[Fix lock dependency between mapping->i_mmap_rwsem and vma->vm_sequence by
using vm_raw_write* functions]
Signed-off-by: Laurent Dufour
---
include/linux/mm.h
When handling page fault without holding the mmap_sem the fetch of the
pte lock pointer and the locking will have to be done while ensuring
that the VMA is not touched in our back.
So move the fetch and locking operations in a dedicated function.
Signed-off-by: Laurent Dufour
---
mm/memory.c
n the file]
Signed-off-by: Laurent Dufour
---
include/linux/mm.h | 1 +
mm/memory.c| 56 ++
2 files changed, 41 insertions(+), 16 deletions(-)
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 4d02524a7998..2f3e98edc94a 100644
---
t for book3e_hugetlb_preload()
called by update_mmu_cache()
Signed-off-by: Laurent Dufour
---
arch/powerpc/Kconfig | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index 73ce5dd07642..acf2696a6505 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kc
Introduce CONFIG_SPECULATIVE_PAGE_FAULT which turns on the Speculative Page
Fault handler when building for 64bits with SMP.
Cc: Thomas Gleixner
Signed-off-by: Laurent Dufour
---
arch/x86/Kconfig | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index
This configuration variable will be used to build the code needed to
handle speculative page fault.
By default it is turned off, and activated depending on architecture
support.
Suggested-by: Thomas Gleixner
Signed-off-by: Laurent Dufour
---
mm/Kconfig | 3 +++
1 file changed, 3 insertions
p_sem.
- In patch 11-12, don't check for vma boundaries when
page_add_new_anon_rmap() is called during the spf path and protect against
anon_vma pointer's update.
- In patch 13-16, add performance events to report number of successful
and failed speculative events.
[1]
http:/
ock:
>
> On 02/16/2018 10:25 AM, Laurent Dufour wrote:
>> +static bool pte_map_lock(struct vm_fault *vmf)
>> +{
> ...snip...
>> + if (!pmd_same(pmdval, vmf->orig_pmd))
>> + goto out;
>
> Since SPF can now call pmd_same without THP, maybe the way
aries matched the passed address and release
the reference on the VMA so that it can be freed if needed.
In the case the VMA is freed, can_reuse_spf_vma() will have returned false
as the VMA is no more in the RB tree.
Signed-off-by: Laurent Dufour
---
include/linux/mm.h | 5 +-
mm/memory.c
multithreaded process as there is no
risk of contention on the mmap_sem otherwise.
Build on if CONFIG_SPECULATIVE_PAGE_FAULT is defined (currently for
BOOK3S_64 && SMP).
Signed-off-by: Laurent Dufour
---
arch/powerpc/mm/fault.c | 31 ++-
1 file changed, 30 in
path in case of retry]
Signed-off-by: Laurent Dufour
---
arch/x86/mm/fault.c | 38 +-
1 file changed, 37 insertions(+), 1 deletion(-)
diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
index 800de815519c..d9f9236ccb9a 100644
--- a/arch/x86/mm/fault.c
+++ b
This patch a set of new trace events to collect the speculative page fault
event failures.
Signed-off-by: Laurent Dufour
---
include/trace/events/pagefault.h | 87
mm/memory.c | 62 ++--
2 files changed, 136
Add support for the new speculative faults event.
Signed-off-by: Laurent Dufour
---
tools/include/uapi/linux/perf_event.h | 1 +
tools/perf/util/evsel.c | 1 +
tools/perf/util/parse-events.c| 4
tools/perf/util/parse-events.l| 1 +
tools/perf/util/python.c
Add a new software event to count succeeded speculative page faults.
Signed-off-by: Laurent Dufour
---
include/uapi/linux/perf_event.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/include/uapi/linux/perf_event.h b/include/uapi/linux/perf_event.h
index e0739a1aa4b2..164383273147 100644
exiting an
in progress speculative page fault.
[1] https://patchwork.kernel.org/patch/5108281/
Cc: Peter Zijlstra (Intel)
Cc: Matthew Wilcox
Signed-off-by: Laurent Dufour
---
include/linux/mm_types.h | 4 ++
kernel/fork.c| 3 ++
mm/init-mm.c | 3 ++
mm/internal.h
Check PMD against concurrent collapsing operation]
[Try spin lock the pte during the speculative path to avoid deadlock with
other CPU's invalidating the TLB and requiring this CPU to catch the
inter processor's interrupt]
Signed-off-by: Laurent Dufour
---
include/linux/hugetlb_inline.h
_vma is still valid as explained
above.
Signed-off-by: Laurent Dufour
---
include/linux/rmap.h | 12 ++--
mm/memory.c | 8
mm/rmap.c| 5 ++---
3 files changed, 16 insertions(+), 9 deletions(-)
diff --git a/include/linux/rmap.h b/include/linux/rmap.h
index 988d
501 - 600 of 924 matches
Mail list logo
