Re: [PATCH v2 19/20] x86/mm: Add speculative pagefault handling

On 29/08/2017 16:50, Laurent Dufour wrote: > On 21/08/2017 09:29, Anshuman Khandual wrote: >> On 08/18/2017 03:35 AM, Laurent Dufour wrote: >>> From: Peter Zijlstra <pet...@infradead.org> >>> >>> Try a speculative fault before acquiring mmap_sem, if i

Re: [PATCH v2 20/20] powerpc/mm: Add speculative page fault

On 21/08/2017 08:58, Anshuman Khandual wrote: > On 08/18/2017 03:35 AM, Laurent Dufour wrote: >> This patch enable the speculative page fault on the PowerPC >> architecture. >> >> This will try a speculative page fault without holding the mmap_sem, >>

[PATCH v5 17/22] mm: Try spin lock in speculative path

async_page_fault+0x28/0x30 Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- mm/memory.c | 19 --- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index eff40abfc1a6..d1278fc15a91 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -

[PATCH v5 14/22] mm: Introduce __vm_normal_page()

value as parameter. Note: The speculative path is turned on for architecture providing support for special PTE flag. So only the first block of vm_normal_page is used during the speculative path. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/mm.h | 7 +

[PATCH v5 04/22] mm: Prepare for FAULT_FLAG_SPECULATIVE

] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/mm.h | 1 + mm/memory.c| 55 ++ 2 files changed, 40 insertions(+), 16 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 3cc40742

[PATCH v5 03/22] mm: Dont assume page-table invariance during faults

Peter Zijlstra (Intel) <pet...@infradead.org> [Remove only if !CONFIG_SPF] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- mm/memory.c | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/mm/memory.c b/mm/memory.c index 6632c9b357c9..b7a9baf3df8a 100644

[PATCH v5 18/22] mm: Adding speculative page fault failure trace events

This patch a set of new trace events to collect the speculative page fault event failures. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/trace/events/pagefault.h | 87 mm/memory.c

[PATCH v5 07/22] mm: Protect VMA modifications using VMA sequence count

using WRITE_ONCE to prevent write to be split and intermediate values to be pushed to other CPUs. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- fs/proc/task_mmu.c | 5 - fs/userfaultfd.c | 17 + mm/khugepaged.c| 3 +++ mm/madvise.c | 6

[PATCH v5 08/22] mm: RCU free VMAs

Rename vma_is_dead() to vma_has_changed() and move its adding to the next patch] [Postpone call to mpol_put() as the policy can be used during the speculative path] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> mm/spf: Fix policy free --- include/linux/mm_types.h | 2 + kernel/fork.

[PATCH v5 06/22] mm: VMA sequence count

4.12 kernel] [Build depends on CONFIG_SPF] [Introduce vm_write_* inline function depending on CONFIG_SPF] [Fix lock dependency between mapping->i_mmap_rwsem and vma->vm_sequence by using vm_raw_write* functions] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> Fix lock

[PATCH v5 10/22] mm: Protect SPF handler against anon_vma changes

sequence counter which is updated in unmap_page_range() before locking the pte, and then in free_pgtables() so when locking the pte the change will be detected. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- mm/memory.c | 4 1 file changed, 4 insertions(+) diff --git

[PATCH v5 15/22] mm: Introduce __page_add_new_anon_rmap()

still valid as explained above. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/rmap.h | 12 ++-- mm/memory.c | 8 mm/rmap.c| 5 ++--- 3 files changed, 16 insertions(+), 9 deletions(-) diff --git a/include/linux/rmap.h b/include/

[PATCH v5 01/22] x86/mm: Define CONFIG_SPF

Introduce CONFIG_SPF which turns on the Speculative Page Fault handler when building for 64bits with SMP. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- arch/x86/Kconfig | 4 1 file changed, 4 insertions(+) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 063f1e

[PATCH v5 02/22] powerpc/mm: Define CONFIG_SPF

() called by update_mmu_cache() Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- arch/powerpc/Kconfig | 4 1 file changed, 4 insertions(+) diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index 809c468edab1..661ba5bcf60e 100644 --- a/arch/powerpc/Kconfig +++

[PATCH v5 05/22] mm: Introduce pte_spinlock for FAULT_FLAG_SPECULATIVE

When handling page fault without holding the mmap_sem the fetch of the pte lock pointer and the locking will have to be done while ensuring that the VMA is not touched in our back. So move the fetch and locking operations in a dedicated function. Signed-off-by: Laurent Dufour <l

[PATCH v5 19/22] perf: Add a speculative page fault sw event

Add a new software event to count succeeded speculative page faults. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/uapi/linux/perf_event.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/uapi/linux/perf_event.h b/include/uapi/linux/perf_event.h

[PATCH v5 00/22] Speculative page faults

net/ [4] http://ck.kolivas.org/apps/kernbench/kernbench-0.50/ [5] https://lwn.net/Articles/725607/ Laurent Dufour (16): x86/mm: Define CONFIG_SPF powerpc/mm: Define CONFIG_SPF mm: Introduce pte_spinlock for FAULT_FLAG_SPECULATIVE mm: Protect VMA modifications using VMA sequence count mm: C

[PATCH v5 16/22] mm: Provide speculative fault infrastructure

t VMA growing up or down] [Move check on vm_sequence just before calling handle_pte_fault()] [Don't build SPF services if !CONFIG_SPF] [Add mem cgroup oom check] [Use use READ_ONCE to access p*d entries] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/hugetlb_

[PATCH v5 13/22] mm: Introduce __maybe_mkwrite()

() service which can be called by passing the value of the vm_flags field. There is no change functional changes expected for the other callers of maybe_mkwrite(). Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/mm.h | 9 +++-- mm/memory.c| 6 +++--- 2

[PATCH v5 12/22] mm: Introduce __lru_cache_add_active_or_unevictable

pointer. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/swap.h | 11 +-- mm/memory.c | 8 mm/swap.c| 12 ++-- 3 files changed, 19 insertions(+), 12 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h

[PATCH v5 11/22] mm/migrate: Pass vm_fault pointer to migrate_misplaced_page()

migrate_misplaced_page() is only called during the page fault handling so it's better to pass the pointer to the struct vm_fault instead of the vma. This way during the speculative page fault path the saved vma->vm_flags could be used. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.i

[PATCH v5 21/22] x86/mm: Add speculative pagefault handling

_ALLOW_RETRY is now done in handle_speculative_fault()] [Retry with usual fault path in the case VM_ERROR is returned by handle_speculative_fault(). This allows signal to be delivered] [Don't build SPF call if !CONFIG_SPF] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- arch/x

[PATCH v5 20/22] perf tools: Add support for the SPF perf event

Add support for the new speculative faults event. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- tools/include/uapi/linux/perf_event.h | 1 + tools/perf/util/evsel.c | 1 + tools/perf/util/parse-events.c| 4 tools/perf/util/parse-events.l

[PATCH v5 22/22] powerpc/mm: Add speculative page fault

(currently for BOOK3S_64 && SMP). Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- arch/powerpc/mm/fault.c | 17 + 1 file changed, 17 insertions(+) diff --git a/arch/powerpc/mm/fault.c b/arch/powerpc/mm/fault.c index 4797d08581ce..c018c2554cc8 100644 --- a/a

[PATCH v5 09/22] mm: Cache some VMA fields in the vm_fault structure

hanges. This patch also set the fields in hugetlb_no_page() and __collapse_huge_page_swapin even if it is not need for the callee. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/mm.h | 6 ++ mm/hugetlb.c | 2 ++ mm/khugepaged.c| 2 ++ mm/memory.c

[PATCH v4 00/20] Speculative page faults

ne [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=da915ad5cf25b5f5d358dd3670c3378d8ae8c03e [3] http://ebizzy.sourceforge.net/ [4] http://ck.kolivas.org/apps/kernbench/kernbench-0.50/ [5] https://lwn.net/Articles/725607/ Laurent Dufour (14): mm: Introduce

[PATCH v4 01/20] mm: Dont assume page-table invariance during faults

Peter Zijlstra (Intel) <pet...@infradead.org> [Remove only if !__HAVE_ARCH_CALL_SPF] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- mm/memory.c | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/mm/memory.c b/mm/memory.c index 6632c9b357c9..4e4fe233d0

[PATCH v4 09/20] mm/migrate: Pass vm_fault pointer to migrate_misplaced_page()

migrate_misplaced_page() is only called during the page fault handling so it's better to pass the pointer to the struct vm_fault instead of the vma. This way during the speculative page fault path the saved vma->vm_flags could be used. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.i

[PATCH v4 05/20] mm: Protect VMA modifications using VMA sequence count

using WRITE_ONCE to prevent write to be split and intermediate values to be pushed to other CPUs. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- fs/proc/task_mmu.c | 5 - fs/userfaultfd.c | 17 + mm/khugepaged.c| 3 +++ mm/madvise.c | 6

[PATCH v4 06/20] mm: RCU free VMAs

Rename vma_is_dead() to vma_has_changed() and move its adding to the next patch] [Postpone call to mpol_put() as the policy can be used during the speculative path] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> mm/spf: Fix policy free --- include/linux/mm_types.h | 2 + kernel/fork.

[PATCH v4 04/20] mm: VMA sequence count

4.12 kernel] [Build depends on __HAVE_ARCH_CALL_SPF] [Introduce vm_write_* inline function depending on __HAVE_ARCH_CALL_SPF] [Fix lock dependency between mapping->i_mmap_rwsem and vma->vm_sequence by using vm_raw_write* functions] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.co

[PATCH v4 15/20] mm: Try spin lock in speculative path

async_page_fault+0x28/0x30 Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- mm/memory.c | 19 --- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 6761e3007500..8abfc0e12e25 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -

[PATCH v4 08/20] mm: Protect SPF handler against anon_vma changes

sequence counter which is updated in unmap_page_range() before locking the pte, and then in free_pgtables() so when locking the pte the change will be detected. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- mm/memory.c | 4 1 file changed, 4 insertions(+) diff --git

[PATCH v4 07/20] mm: Cache some VMA fields in the vm_fault structure

hanges. This patch also set the fields in hugetlb_no_page() and __collapse_huge_page_swapin even if it is not need for the callee. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/mm.h | 6 ++ mm/hugetlb.c | 2 ++ mm/khugepaged.c| 2 ++ mm/memory.c

[PATCH v4 03/20] mm: Introduce pte_spinlock for FAULT_FLAG_SPECULATIVE

When handling page fault without holding the mmap_sem the fetch of the pte lock pointer and the locking will have to be done while ensuring that the VMA is not touched in our back. So move the fetch and locking operations in a dedicated function. Signed-off-by: Laurent Dufour <l

[PATCH v4 13/20] mm: Introduce __page_add_new_anon_rmap()

still valid as explained above. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/rmap.h | 12 ++-- mm/memory.c | 8 mm/rmap.c| 5 ++--- 3 files changed, 16 insertions(+), 9 deletions(-) diff --git a/include/linux/rmap.h b/include/

[PATCH v4 02/20] mm: Prepare for FAULT_FLAG_SPECULATIVE

] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/mm.h | 1 + mm/memory.c| 55 ++ 2 files changed, 40 insertions(+), 16 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 3cc40742

[PATCH v4 16/20] mm: Adding speculative page fault failure trace events

This patch a set of new trace events to collect the speculative page fault event failures. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/trace/events/pagefault.h | 87 mm/memory.c

[PATCH v4 14/20] mm: Provide speculative fault infrastructure

access p*d entries] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/hugetlb_inline.h | 2 +- include/linux/mm.h | 5 + include/linux/pagemap.h| 4 +- mm/internal.h | 16 +++ mm/memory.c| 285 +

[PATCH v4 18/20] perf tools: Add support for the SPF perf event

Add support for the new speculative faults event. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- tools/include/uapi/linux/perf_event.h | 1 + tools/perf/util/evsel.c | 1 + tools/perf/util/parse-events.c| 4 tools/perf/util/parse-events.l

[PATCH v4 12/20] mm: Introduce __vm_normal_page()

value as parameter. Note: The speculative path is turned on for architecture providing support for special PTE flag. So only the first block of vm_normal_page is used during the speculative path. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/mm.h | 7 +

[PATCH v4 19/20] x86/mm: Add speculative pagefault handling

_ALLOW_RETRY is now done in handle_speculative_fault()] [Retry with usual fault path in the case VM_ERROR is returned by handle_speculative_fault(). This allows signal to be delivered] [Don't build SPF call if !__HAVE_ARCH_CALL_SPF] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com

[PATCH v4 17/20] perf: Add a speculative page fault sw event

Add a new software event to count succeeded speculative page faults. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/uapi/linux/perf_event.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/uapi/linux/perf_event.h b/include/uapi/linux/perf_event.h

[PATCH v4 20/20] powerpc/mm: Add speculative page fault

currently because: - require CONFIG_PPC_STD_MMU because checks done in set_access_flags_filter() - require BOOK3S because we can't support for book3e_hugetlb_preload() called by update_mmu_cache() Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- arch/powerpc/include/asm/boo

[PATCH v4 11/20] mm: Introduce __maybe_mkwrite()

() service which can be called by passing the value of the vm_flags field. There is no change functional changes expected for the other callers of maybe_mkwrite(). Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/mm.h | 9 +++-- mm/memory.c| 6 +++--- 2

[PATCH v4 10/20] mm: Introduce __lru_cache_add_active_or_unevictable

pointer. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/swap.h | 11 +-- mm/memory.c | 8 mm/swap.c| 12 ++-- 3 files changed, 19 insertions(+), 12 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h

Re: [PATCH v5 07/22] mm: Protect VMA modifications using VMA sequence count

Hi Andrea, On 02/11/2017 21:08, Andrea Arcangeli wrote: > On Thu, Nov 02, 2017 at 06:25:11PM +0100, Laurent Dufour wrote: >> I think there is some memory barrier missing when the VMA is modified so >> currently the modifications done in the VMA structure may not be written >

Re: [PATCH v5 07/22] mm: Protect VMA modifications using VMA sequence count

Hi Andrea, Thanks for reviewing this series, and sorry for the late answer, I took few days off... On 26/10/2017 12:18, Andrea Arcangeli wrote: > Hello Laurent, > > Message-ID: <7ca80231-fe02-a3a7-84bc-ce81690ea...@intel.com> shows > significant slowdown even for brk/malloc ops both single and

Re: [PATCH v5 07/22] mm: Protect VMA modifications using VMA sequence count

On 02/11/2017 16:16, Laurent Dufour wrote: > Hi Andrea, > > Thanks for reviewing this series, and sorry for the late answer, I took few > days off... > > On 26/10/2017 12:18, Andrea Arcangeli wrote: >> Hello Laurent, >> >> Message-ID: <7ca80231-fe02-

Re: KASAN: use-after-free Read in __do_page_fault

Hi Vlastimil, Sorry for the late answer I got a few day off. On 31/10/2017 14:57, Vlastimil Babka wrote: > +CC Andrea, Thorsten, Linus > > On 10/31/2017 02:20 PM, Vlastimil Babka wrote: >> On 10/31/2017 01:42 PM, Dmitry Vyukov wrote: My vm_area_struct is 192 bytes, could be your layout is

Re: [v5,22/22] powerpc/mm: Add speculative page fault

ill-it-scale.per_process_ops > read2 1204838 -1.7% 1183993 > will-it-scale.per_process_ops > futex1 5017718 -1.6% 4938677 > will-it-scale.per_process_ops > 1408250 -1.0% 1394022 > w

Re: [PATCH v10 12/25] mm: cache some VMA fields in the vm_fault structure

On 03/05/2018 17:42, Minchan Kim wrote: > On Thu, May 03, 2018 at 02:25:18PM +0200, Laurent Dufour wrote: >> On 23/04/2018 09:42, Minchan Kim wrote: >>> On Tue, Apr 17, 2018 at 04:33:18PM +0200, Laurent Dufour wrote: >>>> When handling speculative page fault,

Re: [PATCH v10 06/25] mm: make pte_unmap_same compatible with SPF

On 10/05/2018 18:15, vinayak menon wrote: > On Tue, Apr 17, 2018 at 8:03 PM, Laurent Dufour > <lduf...@linux.vnet.ibm.com> wrote: >> pte_unmap_same() is making the assumption that the page table are still >> around because the mmap_sem is held. >> This is

Re: [PATCH v10 02/25] x86/mm: define ARCH_SUPPORTS_SPECULATIVE_PAGE_FAULT

On 08/05/2018 13:04, Punit Agrawal wrote: > Hi Laurent, > > Laurent Dufour <lduf...@linux.vnet.ibm.com> writes: > >> Set ARCH_SUPPORTS_SPECULATIVE_PAGE_FAULT which turns on the >> Speculative Page Fault handler when building for 64bit. >> >> Cc: Thomas G

Re: [PATCH v10 09/25] mm: protect VMA modifications using VMA sequence count

On 23/04/2018 09:19, Minchan Kim wrote: > On Tue, Apr 17, 2018 at 04:33:15PM +0200, Laurent Dufour wrote: >> The VMA sequence count has been introduced to allow fast detection of >> VMA modification when running a page fault handler without holding >> the mmap_sem. >

Re: [PATCH v10 18/25] mm: provide speculative fault infrastructure

On 15/05/2018 15:09, vinayak menon wrote: > On Tue, Apr 17, 2018 at 8:03 PM, Laurent Dufour > <lduf...@linux.vnet.ibm.com> wrote: >> >> +#ifdef CONFIG_SPECULATIVE_PAGE_FAULT >> + >> +#ifndef __HAVE_ARCH_PTE_SPECIAL >> +/* This is required by vm_normal_pa

Re: [PATCH v10 23/25] mm: add speculative page fault vmstats

On 16/05/2018 04:50, Ganesh Mahendran wrote: > 2018-04-17 22:33 GMT+08:00 Laurent Dufour <lduf...@linux.vnet.ibm.com>: >> Add speculative_pgfault vmstat counter to count successful speculative page >> fault handling. >> >> Also fixing a minor typo in include/li

[PATCH v11 13/26] mm: cache some VMA fields in the vm_fault structure

etlb_no_page() and __collapse_huge_page_swapin even if it is not need for the callee. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/mm.h | 10 -- mm/huge_memory.c | 6 +++--- mm/hugetlb.c | 2 ++ mm/khugepaged.c| 2 ++ mm/me

[PATCH v11 12/26] mm: protect SPF handler against anon_vma changes

sequence counter which is updated in unmap_page_range() before locking the pte, and then in free_pgtables() so when locking the pte the change will be detected. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- mm/memory.c | 4 1 file changed, 4 insertions(+) diff --git

[PATCH v11 09/26] mm: VMA sequence count

a lock dependency warning in mmap_region() when entering the error path] [move sequence initialisation INIT_VMA()] [Review the patch description about unmap_page_range()] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/mm.h | 44 +

[PATCH v11 02/26] x86/mm: define ARCH_SUPPORTS_SPECULATIVE_PAGE_FAULT

Set ARCH_SUPPORTS_SPECULATIVE_PAGE_FAULT which turns on the Speculative Page Fault handler when building for 64bit. Cc: Thomas Gleixner <t...@linutronix.de> Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- arch/x86/Kconfig | 1 + 1 file changed, 1 insertion(+) diff

[PATCH v11 11/26] mm: protect mremap() against SPF hanlder

d by calling vm_raw_write_end() by the callee once the ptes have been moved. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/mm.h | 24 +++- mm/mmap.c | 53 + mm/mremap.c| 13

[PATCH v11 05/26] mm: prepare for FAULT_FLAG_SPECULATIVE

ernel] [move pte_map_lock()'s definition upper in the file] [move the define of FAULT_FLAG_SPECULATIVE later in the series] [review error path in do_swap_page(), do_anonymous_page() and wp_page_copy()] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> ---

[PATCH v11 03/26] powerpc/mm: set ARCH_SUPPORTS_SPECULATIVE_PAGE_FAULT

for book3e_hugetlb_preload() called by update_mmu_cache() Cc: Michael Ellerman <m...@ellerman.id.au> Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- arch/powerpc/Kconfig | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index

[PATCH v11 16/26] mm: introduce __vm_normal_page()

value as parameter. Note: The speculative path is turned on for architecture providing support for special PTE flag. So only the first block of vm_normal_page is used during the speculative path. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/mm.

[PATCH v11 14/26] mm/migrate: Pass vm_fault pointer to migrate_misplaced_page()

igned-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/migrate.h | 4 ++-- mm/memory.c | 2 +- mm/migrate.c| 4 ++-- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/include/linux/migrate.h b/include/linux/migrate.h index f2b4abbca55

[PATCH v11 18/26] mm: protect mm_rb tree with a rwlock

ess speculative page fault. [1] https://patchwork.kernel.org/patch/5108281/ Cc: Peter Zijlstra (Intel) <pet...@infradead.org> Cc: Matthew Wilcox <wi...@infradead.org> Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/mm.h | 1 + include/linux/mm_typ

[PATCH v11 15/26] mm: introduce __lru_cache_add_active_or_unevictable

pointer. Acked-by: David Rientjes <rient...@google.com> Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/swap.h | 10 -- mm/memory.c | 8 mm/swap.c| 6 +++--- 3 files changed, 15 insertions(+), 9 deletions(-) diff --git a/i

[PATCH v11 06/26] mm: introduce pte_spinlock for FAULT_FLAG_SPECULATIVE

When handling page fault without holding the mmap_sem the fetch of the pte lock pointer and the locking will have to be done while ensuring that the VMA is not touched in our back. So move the fetch and locking operations in a dedicated function. Signed-off-by: Laurent Dufour <l

[PATCH v11 04/26] arm64/mm: define ARCH_SUPPORTS_SPECULATIVE_PAGE_FAULT

From: Mahendran Ganesh Set ARCH_SUPPORTS_SPECULATIVE_PAGE_FAULT for arm64. This enables Speculative Page Fault handler. Signed-off-by: Ganesh Mahendran --- arch/arm64/Kconfig | 1 + 1 file changed, 1 insertion(+) diff --git

[PATCH v11 20/26] mm: adding speculative page fault failure trace events

This patch a set of new trace events to collect the speculative page fault event failures. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/trace/events/pagefault.h | 80 mm/memory.c

[PATCH v11 21/26] perf: add a speculative page fault sw event

Add a new software event to count succeeded speculative page faults. Acked-by: David Rientjes <rient...@google.com> Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/uapi/linux/perf_event.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/uapi/linux/pe

[PATCH v11 23/26] mm: add speculative page fault vmstats

Add speculative_pgfault vmstat counter to count successful speculative page fault handling. Also fixing a minor typo in include/linux/vm_event_item.h. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/vm_event_item.h | 3 +++ mm/memory.c | 3 +

[PATCH v11 08/26] mm: introduce INIT_VMA()

inline function. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- fs/exec.c | 2 +- include/linux/mm.h | 5 + kernel/fork.c | 2 +- mm/mmap.c | 10 +- mm/nommu.c | 2 +- 5 files changed, 13 insertions(+), 8 deletions(-) diff --gi

[PATCH v11 07/26] mm: make pte_unmap_same compatible with SPF

is then trapped in cow_user_page(). If VM_FAULT_RETRY is returned, it is passed up to the callers to retry the page fault while holding the mmap_sem. Acked-by: David Rientjes <rient...@google.com> Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/mm.h | 4 +++-

[PATCH v11 17/26] mm: introduce __page_add_new_anon_rmap()

still valid as explained above. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/rmap.h | 12 ++-- mm/memory.c | 8 mm/rmap.c| 5 ++--- 3 files changed, 16 insertions(+), 9 deletions(-) diff --git a/include/linux/rmap.h b/include/

[PATCH v11 00/26] Speculative page faults

-6-Another-go-at-speculative-page-faults-tt965642.html#none [2] https://patchwork.kernel.org/patch/687/ Laurent Dufour (20): mm: introduce CONFIG_SPECULATIVE_PAGE_FAULT x86/mm: define ARCH_SUPPORTS_SPECULATIVE_PAGE_FAULT powerpc/mm: set ARCH_SUPPORTS_SPECULATIVE_PAGE_FAULT mm: introduce

[PATCH v11 01/26] mm: introduce CONFIG_SPECULATIVE_PAGE_FAULT

is not set. Suggested-by: Thomas Gleixner <t...@linutronix.de> Suggested-by: David Rientjes <rient...@google.com> Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- mm/Kconfig | 22 ++ 1 file changed, 22 insertions(+) diff --git a/mm/Kconfig

[PATCH v11 22/26] perf tools: add support for the SPF perf event

Add support for the new speculative faults event. Acked-by: David Rientjes <rient...@google.com> Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- tools/include/uapi/linux/perf_event.h | 1 + tools/perf/util/evsel.c | 1 + tools/perf/util/parse-events.

[PATCH v11 10/26] mm: protect VMA modifications using VMA sequence count

to prevent write to be split and intermediate values to be pushed to other CPUs. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- fs/proc/task_mmu.c | 5 - fs/userfaultfd.c | 17 + mm/khugepaged.c| 3 +++ mm/madvise.c | 6 +- mm/mempo

[PATCH v11 19/26] mm: provide speculative fault infrastructure

d a check against mm->mm_users in handle_speculative_fault() defined in mm.h] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/hugetlb_inline.h | 2 +- include/linux/mm.h | 30 include/linux/pagemap.h| 4 +- mm/internal.

[PATCH v11 25/26] powerpc/mm: add speculative page fault

for multithreaded process as there is no risk of contention on the mmap_sem otherwise. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- arch/powerpc/mm/fault.c | 16 1 file changed, 16 insertions(+) diff --git a/arch/powerpc/mm/fault.c b/arch/powerpc/mm/fault.c

[PATCH v11 26/26] arm64/mm: add speculative page fault

ds, 15% or even more improvement can be got. Signed-off-by: Ganesh Mahendran <opensource.gan...@gmail.com> [handle_speculative_fault() is no more returning the vma pointer] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- arch/arm64/mm/fault.c | 12 1 file chan

[PATCH v11 24/26] x86/mm: add speculative pagefault handling

_ALLOW_RETRY is now done in handle_speculative_fault()] [Retry with usual fault path in the case VM_ERROR is returned by handle_speculative_fault(). This allows signal to be delivered] [Don't build SPF call if !CONFIG_SPECULATIVE_PAGE_FAULT] [Handle memory protection key fault] Signed-off-by: La

Re: [PATCH v11 01/26] mm: introduce CONFIG_SPECULATIVE_PAGE_FAULT

On 17/05/2018 18:36, Randy Dunlap wrote: > Hi, > > On 05/17/2018 04:06 AM, Laurent Dufour wrote: >> This configuration variable will be used to build the code needed to >> handle speculative page fault. >> >> By default it is turned off, and activated dep

Re: [PATCH v11 01/26] mm: introduce CONFIG_SPECULATIVE_PAGE_FAULT

On 17/05/2018 19:19, Matthew Wilcox wrote: > On Thu, May 17, 2018 at 09:36:00AM -0700, Randy Dunlap wrote: >>> +If the speculative page fault fails because of a concurrency is >> >> because a concurrency is > > While one can use concurrency as a noun,

[FIX PATCH v11 01/26] mm: introduce CONFIG_SPECULATIVE_PAGE_FAULT

is not set. Suggested-by: Thomas Gleixner <t...@linutronix.de> Suggested-by: David Rientjes <rient...@google.com> Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- mm/Kconfig | 22 ++ 1 file changed, 22 insertions(+) diff --git a/mm/Kconfig

Re: [PATCH v11 00/26] Speculative page faults

quot; while responding to the v11 header series... Were these tests done on v9 or v11 ? Cheers, Laurent. > > The regression result is sorted by the metric will-it-scale.per_thread_ops. > Branch: Laurent-Dufour/Speculative-page-faults/20180316-151833 (V9 patch > seri

Re: [RFC v3 PATCH 5/5] x86: check VM_DEAD flag in page fault

On 30/06/2018 00:39, Yang Shi wrote: > Check VM_DEAD flag of vma in page fault handler, if it is set, trigger > SIGSEGV. > > Cc: Michal Hocko > Cc: Thomas Gleixner > Cc: Ingo Molnar > Cc: "H. Peter Anvin" > Signed-off-by: Yang Shi > --- > arch/x86/mm/fault.c | 4 > 1 file changed, 4

Re: [RFC v3 PATCH 5/5] x86: check VM_DEAD flag in page fault

On 02/07/2018 14:15, Michal Hocko wrote: > On Mon 02-07-18 10:45:03, Laurent Dufour wrote: >> On 30/06/2018 00:39, Yang Shi wrote: >>> Check VM_DEAD flag of vma in page fault handler, if it is set, trigger >>> SIGSEGV. >>> >>> Cc: Michal Hocko >>

Re: [RFC v3 PATCH 5/5] x86: check VM_DEAD flag in page fault

On 02/07/2018 14:45, Michal Hocko wrote: > On Mon 02-07-18 14:26:09, Laurent Dufour wrote: >> On 02/07/2018 14:15, Michal Hocko wrote: >>> On Mon 02-07-18 10:45:03, Laurent Dufour wrote: >>>> On 30/06/2018 00:39, Yang Shi wrote: >>>>> Ch

Re: [PATCH v10 17/25] mm: protect mm_rb tree with a rwlock

Hi Punit, Thanks for reviewing this series. On 30/04/2018 20:47, Punit Agrawal wrote: > Hi Laurent, > > One nitpick below. > > On Tue, Apr 17, 2018 at 3:33 PM, Laurent Dufour > <lduf...@linux.vnet.ibm.com> wrote: >> This change is inspired by the Pe

Re: [PATCH 1/2] arm64/mm: define ARCH_SUPPORTS_SPECULATIVE_PAGE_FAULT

On 02/05/2018 09:54, Ganesh Mahendran wrote: > Set ARCH_SUPPORTS_SPECULATIVE_PAGE_FAULT for arm64. This > enables Speculative Page Fault handler. > > Signed-off-by: Ganesh Mahendran > --- > This patch is on top of Laurent's v10 spf > --- > arch/arm64/Kconfig | 1 + >

Re: [PATCH 2/2] arm64/mm: add speculative page fault

On 02/05/2018 09:54, Ganesh Mahendran wrote: > This patch enables the speculative page fault on the arm64 > architecture. > > I completed spf porting in 4.9. From the test result, > we can see app launching time improved by about 10% in average. > For the apps which have more than 50 threads, 15%

Re: [PATCH v10 00/25] Speculative page faults

On 02/05/2018 16:17, Punit Agrawal wrote: > Hi Laurent, > > One query below - > > Laurent Dufour <lduf...@linux.vnet.ibm.com> writes: > > [...] > >> >> Ebizzy: >> --- >> The test is counting the number of records per second i

Re: [PATCH v10 08/25] mm: VMA sequence count

On 23/04/2018 08:42, Minchan Kim wrote: > On Tue, Apr 17, 2018 at 04:33:14PM +0200, Laurent Dufour wrote: >> From: Peter Zijlstra <pet...@infradead.org> >> >> Wrap the VMA modifications (vma_adjust/unmap_page_range) with sequence >> counts such that we ca

Re: [PATCH v10 06/25] mm: make pte_unmap_same compatible with SPF

On 23/04/2018 08:31, Minchan Kim wrote: > On Tue, Apr 17, 2018 at 04:33:12PM +0200, Laurent Dufour wrote: >> pte_unmap_same() is making the assumption that the page table are still >> around because the mmap_sem is held. >> This is no more the case when running a s

Re: [PATCH v10 08/25] mm: VMA sequence count

On 01/05/2018 15:16, Minchan Kim wrote: > On Mon, Apr 30, 2018 at 05:14:27PM +0200, Laurent Dufour wrote: >> >> >> On 23/04/2018 08:42, Minchan Kim wrote: >>> On Tue, Apr 17, 2018 at 04:33:14PM +0200, Laurent Dufour wrote: >>>> From: Peter Zijlstra <

Re: [PATCH v10 24/25] x86/mm: add speculative pagefault handling

umping in bit late to the discussion. Hi Punit, Thanks for giving this series a review. I don't have arm64 hardware to play with, but I'll be happy to add arm64 patches to my series and to try to maintain them. > > On Tue, Apr 17, 2018 at 3:33 PM, Laurent Dufour > <lduf...@linux.vnet.ibm.com&g

Re: [PATCH v10 12/25] mm: cache some VMA fields in the vm_fault structure

On 23/04/2018 09:42, Minchan Kim wrote: > On Tue, Apr 17, 2018 at 04:33:18PM +0200, Laurent Dufour wrote: >> When handling speculative page fault, the vma->vm_flags and >> vma->vm_page_prot fields are read once the page table lock is released. So >> there is no more

<    1   2   3   4   5   6   7   8   9   10   >