Re: [PATCH v2 4/6] mm: replace vma->vm_flags indirect modification in ksm_madvise

On Wed, Jan 25, 2023 at 1:38 AM 'Michal Hocko' via kernel-team wrote: > > On Wed 25-01-23 00:38:49, Suren Baghdasaryan wrote: > > Replace indirect modifications to vma->vm_flags with calls to modifier > > functions to be able to track flag changes and to keep vma locking

[PATCH v3 1/7] kernel/fork: convert vma assignment to a memcpy

Convert vma assignment in vm_area_dup() to a memcpy() to prevent compiler errors when we add a const modifier to vma->vm_flags. Signed-off-by: Suren Baghdasaryan --- kernel/fork.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/fork.c b/kernel/fork.c in

[PATCH v3 2/7] mm: introduce vma->vm_flags wrapper functions

are updated. This way we can better check and control correct locking behavior during these updates. Signed-off-by: Suren Baghdasaryan --- include/linux/mm.h | 37 + include/linux/mm_types.h | 10 +- 2 files changed, 46 insertions(+), 1 deletion

Re: [PATCH v2 4/6] mm: replace vma->vm_flags indirect modification in ksm_madvise

On Wed, Jan 25, 2023 at 9:08 AM Michal Hocko wrote: > > On Wed 25-01-23 08:57:48, Suren Baghdasaryan wrote: > > On Wed, Jan 25, 2023 at 1:38 AM 'Michal Hocko' via kernel-team > > wrote: > > > > > > On Wed 25-01-23 00:38:49, Suren Baghdasaryan wrote: > >

[PATCH v3 6/7] mm: introduce mod_vm_flags_nolock and use it in untrack_pfn

it is not required. Pass a hint to untrack_pfn to conditionally use mod_vm_flags_nolock for flags modification and to avoid assertion. Signed-off-by: Suren Baghdasaryan --- arch/x86/mm/pat/memtype.c | 10 +++--- include/linux/mm.h| 16 +--- include/linux/pgtable.h | 5 +++-- mm/memory.c

[PATCH v3 5/7] mm: replace vma->vm_flags indirect modification in ksm_madvise

Replace indirect modifications to vma->vm_flags with calls to modifier functions to be able to track flag changes and to keep vma locking correctness. Signed-off-by: Suren Baghdasaryan Acked-by: Michal Hocko --- arch/powerpc/kvm/book3s_hv_uvmem.c | 5 - arch/s390/mm/gma

Re: [PATCH v3 5/7] mm: replace vma->vm_flags indirect modification in ksm_madvise

On Thu, Jan 26, 2023 at 7:19 AM Mel Gorman wrote: > > On Wed, Jan 25, 2023 at 03:35:52PM -0800, Suren Baghdasaryan wrote: > > Replace indirect modifications to vma->vm_flags with calls to modifier > > functions to be able to track flag changes and to keep vma lo

Re: [PATCH v3 6/7] mm: introduce mod_vm_flags_nolock and use it in untrack_pfn

On Thu, Jan 26, 2023 at 7:47 AM Mel Gorman wrote: > > On Wed, Jan 25, 2023 at 03:35:53PM -0800, Suren Baghdasaryan wrote: > > In cases when VMA flags are modified after VMA was isolated and mmap_lock > > was downgraded, flags modifications would result in an assertion because &

Re: [PATCH v3 4/7] mm: replace vma->vm_flags direct modifications with modifier calls

On Thu, Jan 26, 2023 at 9:27 AM Mel Gorman wrote: > > On Thu, Jan 26, 2023 at 08:10:26AM -0800, Suren Baghdasaryan wrote: > > On Thu, Jan 26, 2023 at 7:10 AM Mel Gorman > > wrote: > > > > > > On Wed, Jan 25, 2023 at 03:35:51PM -0800, Suren Baghda

Re: [PATCH v3 6/7] mm: introduce mod_vm_flags_nolock and use it in untrack_pfn

On Thu, Jan 26, 2023 at 9:32 AM Mel Gorman wrote: > > On Thu, Jan 26, 2023 at 08:18:31AM -0800, Suren Baghdasaryan wrote: > > On Thu, Jan 26, 2023 at 7:47 AM Mel Gorman > > wrote: > > > > > > On Wed, Jan 25, 2023 at 03:35:53PM -0800, Suren Baghdasaryan wr

Re: [PATCH v3 2/7] mm: introduce vma->vm_flags wrapper functions

On Thu, Jan 26, 2023 at 5:58 AM Mel Gorman wrote: > > On Wed, Jan 25, 2023 at 03:35:49PM -0800, Suren Baghdasaryan wrote: > > vm_flags are among VMA attributes which affect decisions like VMA merging > > and splitting. Therefore all vm_flags modifications are performed after &g

Re: [PATCH v3 1/7] kernel/fork: convert vma assignment to a memcpy

On Thu, Jan 26, 2023 at 3:52 AM Mel Gorman wrote: > > On Wed, Jan 25, 2023 at 05:34:49PM -0800, Andrew Morton wrote: > > On Wed, 25 Jan 2023 16:50:01 -0800 Suren Baghdasaryan > > wrote: > > > > > On Wed, Jan 25, 2023 at 4:22 PM Andrew Morton > > > wro

Re: [PATCH v3 4/7] mm: replace vma->vm_flags direct modifications with modifier calls

On Thu, Jan 26, 2023 at 7:10 AM Mel Gorman wrote: > > On Wed, Jan 25, 2023 at 03:35:51PM -0800, Suren Baghdasaryan wrote: > > Replace direct modifications to vma->vm_flags with calls to modifier > > functions to be able to track flag changes and to keep vma locking > >

[PATCH v4 3/7] mm: replace VM_LOCKED_CLEAR_MASK with VM_LOCKED_MASK

To simplify the usage of VM_LOCKED_CLEAR_MASK in vm_flags_clear(), replace it with VM_LOCKED_MASK bitmask and convert all users. Signed-off-by: Suren Baghdasaryan Acked-by: Michal Hocko Acked-by: Mel Gorman Acked-by: Mike Rapoport (IBM) --- include/linux/mm.h | 4 ++-- kernel/fork.c | 2

[PATCH v4 0/7] introduce vm_flags modifier functions

...@google.com/ [2] https://lore.kernel.org/lkml/20230125233554.153109-1-sur...@google.com/ Suren Baghdasaryan (7): kernel/fork: convert vma assignment to a memcpy mm: introduce vma->vm_flags wrapper functions mm: replace VM_LOCKED_CLEAR_MASK with VM_LOCKED_MASK mm: replace vma->vm_flags

[PATCH v4 1/7] kernel/fork: convert vma assignment to a memcpy

Convert vma assignment in vm_area_dup() to a memcpy() to prevent compiler errors when we add a const modifier to vma->vm_flags. Signed-off-by: Suren Baghdasaryan Acked-by: Mel Gorman --- kernel/fork.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/fork.c b/ker

[PATCH v4 4/7] mm: replace vma->vm_flags direct modifications with modifier calls

Replace direct modifications to vma->vm_flags with calls to modifier functions to be able to track flag changes and to keep vma locking correctness. Signed-off-by: Suren Baghdasaryan Acked-by: Michal Hocko Acked-by: Mel Gorman Acked-by: Mike Rapoport (IBM) Acked-by: Sebastian Reic

[PATCH v4 5/7] mm: replace vma->vm_flags indirect modification in ksm_madvise

Replace indirect modifications to vma->vm_flags with calls to modifier functions to be able to track flag changes and to keep vma locking correctness. Signed-off-by: Suren Baghdasaryan Acked-by: Michal Hocko Acked-by: Mel Gorman Acked-by: Mike Rapoport (IBM) --- arch/powerpc/

[PATCH v4 2/7] mm: introduce vma->vm_flags wrapper functions

are updated. This way we can better check and control correct locking behavior during these updates. Signed-off-by: Suren Baghdasaryan Reviewed-by: Davidlohr Bueso Acked-by: Michal Hocko Acked-by: Mel Gorman --- include/linux/mm.h | 40 include

[PATCH v4 7/7] mm: export dump_mm()

mmap_assert_write_locked() is used in vm_flags modifiers. Because mmap_assert_write_locked() uses dump_mm() and vm_flags are sometimes modified from inside a module, it's necessary to export dump_mm() function. Signed-off-by: Suren Baghdasaryan Acked-by: Michal Hocko Acked-by: Mike Rapoport

[PATCH v4 6/7] mm: introduce __vm_flags_mod and use it in untrack_pfn

responsibility for the required locking. Pass a hint to untrack_pfn to conditionally use __vm_flags_mod for flags modification to avoid assertion. Signed-off-by: Suren Baghdasaryan Acked-by: Michal Hocko --- arch/x86/mm/pat/memtype.c | 10 +++--- include/linux/mm.h| 14

Re: [PATCH v2 1/6] mm: introduce vma->vm_flags modifier functions

On Thu, Jan 26, 2023 at 7:09 AM Matthew Wilcox wrote: > > On Thu, Jan 26, 2023 at 04:50:59PM +0200, Mike Rapoport wrote: > > On Thu, Jan 26, 2023 at 11:17:09AM +0200, Mike Rapoport wrote: > > > On Wed, Jan 25, 2023 at 12:38:46AM -0800, Suren Baghdasaryan wrote: >

[PATCH v2 21/33] kernel/fork: assert no VMA readers during its destruction

Assert there are no holders of VMA lock for reading when it is about to be destroyed. Signed-off-by: Suren Baghdasaryan --- kernel/fork.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/kernel/fork.c b/kernel/fork.c index 9141427a98b2..a08cc0e2bfde 100644 --- a/kernel/fork.c +++ b/kernel

[PATCH v2 22/33] mm/mmap: prevent pagefault handler from racing with mmu_notifier registration

order as in page fault handlers. Signed-off-by: Suren Baghdasaryan --- mm/mmap.c | 9 + 1 file changed, 9 insertions(+) diff --git a/mm/mmap.c b/mm/mmap.c index 3baf218836bb..3d0cfbc92745 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -3501,6 +3501,7 @@ static void vm_lock_mapping(struct

[PATCH v2 33/33] mm: separate vma->lock from vm_area_struct

k anymore. A number of drivers allocate a pseudo VMA on the stack but they never use the VMA's lock, therefore it does not need to be allocated. The future drivers which might need the VMA lock should use vm_area_alloc()/vm_area_free() to allocate the VMA. Signed-off-by: Suren Baghdasaryan --- incl

[PATCH v2 04/33] maple_tree: remove extra smp_wmb() from mas_dead_leaves()

From: Liam Howlett The call to mte_set_dead_node() before the smp_wmb() already calls smp_wmb() so this is not needed. This is an optimization for the RCU mode of the maple tree. Fixes: 54a611b60590 ("Maple Tree: add new data structure") Signed-off-by: Liam Howlett Signed-off

[PATCH v2 09/33] mm: rcu safe VMA freeing

From: Michel Lespinasse This prepares for page faults handling under VMA lock, looking up VMAs under protection of an rcu read lock, instead of the usual mmap read lock. Signed-off-by: Michel Lespinasse Signed-off-by: Suren Baghdasaryan --- include/linux/mm_types.h | 13 ++--- kernel

[PATCH v2 10/33] mm: move mmap_lock assert function definitions

Move mmap_lock assert function definitions up so that they can be used by other mmap_lock routines. Signed-off-by: Suren Baghdasaryan --- include/linux/mmap_lock.h | 24 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/include/linux/mmap_lock.h b/include

[PATCH v2 19/33] mm: conditionally write-lock VMA in free_pgtables

to free_pgtables for such scenario. Signed-off-by: Suren Baghdasaryan --- mm/internal.h | 2 +- mm/memory.c | 6 +- mm/mmap.c | 5 +++-- 3 files changed, 9 insertions(+), 4 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index 90bb2078444c..52d7e9c2e58f 100644 --- a/mm/internal.h +++ b/mm

[PATCH v2 31/33] powerc/mm: try VMA lock-based page fault handling first

From: Laurent Dufour Attempt VMA lock-based page fault handling first, and fall back to the existing mmap_lock-based handling if that fails. Copied from "x86/mm: try VMA lock-based page fault handling first" Signed-off-by: Laurent Dufour Signed-off-by: Suren Baghdasaryan --- arch/

Re: [PATCH v4 3/7] mm: replace VM_LOCKED_CLEAR_MASK with VM_LOCKED_MASK

On Fri, Jan 27, 2023 at 10:12 AM Davidlohr Bueso wrote: > > On Thu, 26 Jan 2023, Suren Baghdasaryan wrote: > > >To simplify the usage of VM_LOCKED_CLEAR_MASK in vm_flags_clear(), > >replace it with VM_LOCKED_MASK bitmask and convert all users. > > Might be good to m

[PATCH v2 02/33] maple_tree: Detect dead nodes in mas_start()

t;) Signed-off-by: Liam Howlett Signed-off-by: Suren Baghdasaryan --- lib/maple_tree.c | 4 1 file changed, 4 insertions(+) diff --git a/lib/maple_tree.c b/lib/maple_tree.c index e5eeecd14eee..482e17a460cb 100644 --- a/lib/maple_tree.c +++ b/lib/maple_tree.c @@ -1353,12 +1353,16 @@ sta

[PATCH v2 03/33] maple_tree: Fix freeing of nodes in rcu mode

Tree: add new data structure") Signed-off-by: Liam Howlett Signed-off-by: Suren Baghdasaryan --- lib/maple_tree.c | 73 1 file changed, 62 insertions(+), 11 deletions(-) diff --git a/lib/maple_tree.c b/lib/maple_tree.c index 482e17a460cb..73

[PATCH v2 13/33] mm/mmap: move VMA locking before vma_adjust_trans_huge call

vma_adjust_trans_huge() modifies the VMA and such modifications should be done after VMA is marked as being written. Therefore move VMA flag modifications before vma_adjust_trans_huge() so that VMA is marked before all these modifications. Signed-off-by: Suren Baghdasaryan --- mm/mmap.c | 3

[PATCH v2 18/33] mm: write-lock VMAs before removing them from VMA tree

Write-locking VMAs before isolating them ensures that page fault handlers don't operate on isolated VMAs. Signed-off-by: Suren Baghdasaryan --- mm/mmap.c | 1 + mm/nommu.c | 5 + 2 files changed, 6 insertions(+) diff --git a/mm/mmap.c b/mm/mmap.c index b3c247073aa0..5bdfd087b632 100644

[PATCH v2 17/33] mm/mremap: write-lock VMA while remapping it to a new address range

Write-lock VMA as locked before copying it and when copy_vma produces a new VMA. Signed-off-by: Suren Baghdasaryan Reviewed-by: Laurent Dufour --- mm/mmap.c | 1 + mm/mremap.c | 1 + 2 files changed, 2 insertions(+) diff --git a/mm/mmap.c b/mm/mmap.c index 60038c24d836..b3c247073aa0 100644

[PATCH v2 24/33] mm: fall back to mmap_lock if vma->anon_vma is not yet set

ed-off-by: Suren Baghdasaryan --- mm/memory.c | 4 1 file changed, 4 insertions(+) diff --git a/mm/memory.c b/mm/memory.c index 5568fcb0a46b..593548f24007 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5244,6 +5244,10 @@ struct vm_area_struct *lock_vma_under_rcu(struct mm_stru

[PATCH v2 26/33] mm: prevent do_swap_page from handling page faults under VMA lock

Due to the possibility of do_swap_page dropping mmap_lock, abort fault handling under VMA lock and retry holding mmap_lock. This can be handled more gracefully in the future. Signed-off-by: Suren Baghdasaryan Reviewed-by: Laurent Dufour --- mm/memory.c | 5 + 1 file changed, 5 insertions

[PATCH v2 32/33] mm/mmap: free vm_area_struct without call_rcu in exit_mmap

caused by it. Signed-off-by: Suren Baghdasaryan --- include/linux/mm.h | 2 ++ kernel/fork.c | 2 +- mm/mmap.c | 11 +++ 3 files changed, 10 insertions(+), 5 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index a6de58bb40c7..1c4ddcd6fd84 100644

[PATCH v2 05/33] maple_tree: Fix write memory barrier of nodes once dead for RCU mode

e RCU mode of the maple tree. Fixes: 54a611b60590 ("Maple Tree: add new data structure") Signed-off-by: Liam R. Howlett Signed-off-by: Suren Baghdasaryan --- lib/maple_tree.c | 7 +-- tools/testing/radix-tree/maple.c | 16 2 files changed, 21 insert

[PATCH v2 06/33] maple_tree: Add smp_rmb() to dead node detection

re") Signed-off-by: Liam R. Howlett Signed-off-by: Suren Baghdasaryan --- lib/maple_tree.c | 8 ++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/lib/maple_tree.c b/lib/maple_tree.c index 49e399e8afaa..859303d2da90 100644 --- a/lib/maple_tree.c +++ b/lib/maple_tree.c @@ -532

[PATCH v2 14/33] mm/khugepaged: write-lock VMA while collapsing a huge page

a fourth lock under which page tables can be traversed, and so khugepaged must also lock out that one. Signed-off-by: Suren Baghdasaryan --- mm/khugepaged.c | 5 + mm/rmap.c | 31 --- 2 files changed, 21 insertions(+), 15 deletions(-) diff --git a/mm

[PATCH v2 23/33] mm: introduce lock_vma_under_rcu to be used from arch-specific code

is acquired. VMA lock statistics are updated according to the results. For now only anonymous VMAs can be searched this way. In other cases the function returns NULL. Signed-off-by: Suren Baghdasaryan --- include/linux/mm.h | 3 +++ mm/memory.c| 51

[PATCH v2 07/33] mm: Enable maple tree RCU mode by default.

to ensure the nodes remain valid for readers. Signed-off-by: Liam R. Howlett Signed-off-by: Suren Baghdasaryan --- include/linux/mm_types.h | 3 ++- kernel/fork.c| 3 +++ mm/mmap.c| 4 +++- 3 files changed, 8 insertions(+), 2 deletions(-) diff --git a/include/linux/mm

[PATCH v2 08/33] mm: introduce CONFIG_PER_VMA_LOCK

needs modifications to handle faults under VMA lock. Signed-off-by: Suren Baghdasaryan --- mm/Kconfig | 12 1 file changed, 12 insertions(+) diff --git a/mm/Kconfig b/mm/Kconfig index ca98b2072df5..2e4a7e61768a 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -1211,6 +1211,18 @@ config

[PATCH v2 15/33] mm/mmap: write-lock VMAs before merging, splitting or expanding them

during COW operation. Write-lock all VMAs which might be affected by a merge or split operation before making decision how such operations should be performed. Signed-off-by: Suren Baghdasaryan --- mm/mmap.c | 23 --- 1 file changed, 20 insertions(+), 3 deletions(-) diff --gi

[PATCH v2 16/33] mm/mmap: write-lock VMA before shrinking or expanding it

vma_expand and vma_shrink change VMA boundaries. Expansion might also result in freeing of an adjacent VMA. Write-lock affected VMAs to prevent concurrent page faults. Signed-off-by: Suren Baghdasaryan --- mm/mmap.c | 5 + 1 file changed, 5 insertions(+) diff --git a/mm/mmap.c b/mm/mmap.c

[PATCH v2 28/33] mm: introduce per-VMA lock statistics

Add a new CONFIG_PER_VMA_LOCK_STATS config option to dump extra statistics about handling page fault under VMA lock. Signed-off-by: Suren Baghdasaryan --- include/linux/vm_event_item.h | 6 ++ include/linux/vmstat.h| 6 ++ mm/Kconfig.debug | 7 +++ mm/vmstat.c

[PATCH v2 27/33] mm: prevent userfaults to be handled under per-vma lock

Due to the possibility of handle_userfault dropping mmap_lock, avoid fault handling under VMA lock and retry holding mmap_lock. This can be handled more gracefully in the future. Signed-off-by: Suren Baghdasaryan Suggested-by: Peter Xu --- mm/memory.c | 9 + 1 file changed, 9

[PATCH v2 00/33] Per-VMA locks

ree: Add smp_rmb() to dead node detection mm: Enable maple tree RCU mode by default. Michel Lespinasse (1): mm: rcu safe VMA freeing Suren Baghdasaryan (24): mm: introduce CONFIG_PER_VMA_LOCK mm: move mmap_lock assert function definitions mm: add per-VMA lock and helper functions to control i

[PATCH v2 25/33] mm: add FAULT_FLAG_VMA_LOCK flag

Add a new flag to distinguish page faults handled under protection of per-vma lock. Signed-off-by: Suren Baghdasaryan Reviewed-by: Laurent Dufour --- include/linux/mm.h | 3 ++- include/linux/mm_types.h | 1 + 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/include/linux

[PATCH v2 01/33] maple_tree: Be more cautious about dead nodes

am Howlett Signed-off-by: Suren Baghdasaryan --- lib/maple_tree.c | 52 +++- 1 file changed, 43 insertions(+), 9 deletions(-) diff --git a/lib/maple_tree.c b/lib/maple_tree.c index 5e9703189259..e5eeecd14eee 100644 --- a/lib/maple_tree.c +++ b/lib/ma

[PATCH v2 11/33] mm: add per-VMA lock and helper functions to control it

ive mmap_lock is released by incrementing mm sequence counter (mm_lock_seq). - write downgrade - if the mmap_lock is downgraded to the read lock, all vma write locks are released as well (effectivelly same as write unlock). Signed-off-by: Suren Baghdasaryan --- include/linux/m

[PATCH v2 12/33] mm: mark VMA as being written when changing vm_flags

Updates to vm_flags have to be done with VMA marked as being written for preventing concurrent page faults or other modifications. Signed-off-by: Suren Baghdasaryan --- include/linux/mm.h | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/include/linux/mm.h b/include

[PATCH v2 20/33] mm/mmap: write-lock adjacent VMAs if they can grow into unmapped area

While unmapping VMAs, adjacent VMAs might be able to grow into the area being unmapped. In such cases write-lock adjacent VMAs to prevent this growth. Signed-off-by: Suren Baghdasaryan --- mm/mmap.c | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/mm/mmap.c b/mm

[PATCH v2 29/33] x86/mm: try VMA lock-based page fault handling first

Attempt VMA lock-based page fault handling first, and fall back to the existing mmap_lock-based handling if that fails. Signed-off-by: Suren Baghdasaryan --- arch/x86/Kconfig| 1 + arch/x86/mm/fault.c | 36 2 files changed, 37 insertions(+) diff --git

[PATCH v2 30/33] arm64/mm: try VMA lock-based page fault handling first

Attempt VMA lock-based page fault handling first, and fall back to the existing mmap_lock-based handling if that fails. Signed-off-by: Suren Baghdasaryan --- arch/arm64/Kconfig| 1 + arch/arm64/mm/fault.c | 36 2 files changed, 37 insertions(+) diff

Re: [PATCH v2 00/33] Per-VMA locks

On Fri, Jan 27, 2023 at 3:26 PM Matthew Wilcox wrote: > > On Fri, Jan 27, 2023 at 02:51:38PM -0800, Andrew Morton wrote: > > On Fri, 27 Jan 2023 11:40:37 -0800 Suren Baghdasaryan > > wrote: > > > > > Per-vma locks idea that was discussed during SPF [1] discus

Re: [PATCH 18/41] mm/khugepaged: write-lock VMA while collapsing a huge page

On Wed, Jan 18, 2023 at 1:40 AM Michal Hocko wrote: > > On Tue 17-01-23 21:28:06, Jann Horn wrote: > > On Tue, Jan 17, 2023 at 4:25 PM Michal Hocko wrote: > > > On Mon 09-01-23 12:53:13, Suren Baghdasaryan wrote: > > > > Protect VMA from concurrent page faul

Re: [PATCH 12/41] mm: add per-VMA lock and helper functions to control it

> > > > > > On Mon, Jan 9, 2023 at 9:54 PM Suren Baghdasaryan > > > > wrote: > > > > > Introduce a per-VMA rw_semaphore to be used during page fault handling > > > > > instead of mmap_lock. Because there are cases when multiple VMAs need &g

Re: [PATCH 39/41] kernel/fork: throttle call_rcu() calls in vm_area_free

On Wed, Jan 18, 2023 at 10:34 AM Paul E. McKenney wrote: > > On Wed, Jan 18, 2023 at 10:04:39AM -0800, Suren Baghdasaryan wrote: > > On Wed, Jan 18, 2023 at 1:49 AM Michal Hocko wrote: > > > > > > On Tue 17-01-23 17:19:46, Suren Baghdasaryan wrote: > > > &

Re: [PATCH 26/41] kernel/fork: assert no VMA readers during its destruction

On Wed, Jan 18, 2023 at 1:43 AM 'Michal Hocko' via kernel-team wrote: > > On Tue 17-01-23 17:53:00, Suren Baghdasaryan wrote: > > On Tue, Jan 17, 2023 at 7:42 AM 'Michal Hocko' via kernel-team > > wrote: > > > > > > On Mon 09-01-23 12:53:21, Sure

Re: [PATCH 27/41] mm/mmap: prevent pagefault handler from racing with mmu_notifier registration

On Wed, Jan 18, 2023 at 4:51 AM Jann Horn wrote: > > On Mon, Jan 9, 2023 at 9:54 PM Suren Baghdasaryan wrote: > > Page fault handlers might need to fire MMU notifications while a new > > notifier is being registered. Modify mm_take_all_locks to write-lock all > > VM

Re: [PATCH 39/41] kernel/fork: throttle call_rcu() calls in vm_area_free

On Wed, Jan 18, 2023 at 1:49 AM Michal Hocko wrote: > > On Tue 17-01-23 17:19:46, Suren Baghdasaryan wrote: > > On Tue, Jan 17, 2023 at 7:57 AM Michal Hocko wrote: > > > > > > On Mon 09-01-23 12:53:34, Suren Baghdasaryan wrote: > > > > call_rcu() can

Re: [PATCH 17/41] mm/mmap: move VMA locking before anon_vma_lock_write call

On Wed, Jan 18, 2023 at 1:23 AM Michal Hocko wrote: > > On Tue 17-01-23 18:01:01, Suren Baghdasaryan wrote: > > On Tue, Jan 17, 2023 at 7:16 AM Michal Hocko wrote: > > > > > > On Mon 09-01-23 12:53:12, Suren Baghdasaryan wrote: > > > > Move VMA flag m

Re: [PATCH 12/41] mm: add per-VMA lock and helper functions to control it

On Wed, Jan 18, 2023 at 1:28 PM Michal Hocko wrote: > > On Wed 18-01-23 09:36:44, Suren Baghdasaryan wrote: > > On Wed, Jan 18, 2023 at 7:11 AM 'Michal Hocko' via kernel-team > > wrote: > > > > > > On Wed 18-01-23 14:23:32, Jann Horn wrote: > > > &

Re: [PATCH 28/41] mm: introduce lock_vma_under_rcu to be used from arch-specific code

On Tue, Jan 17, 2023 at 6:44 PM Matthew Wilcox wrote: > > On Tue, Jan 17, 2023 at 05:06:57PM -0800, Suren Baghdasaryan wrote: > > On Tue, Jan 17, 2023 at 7:47 AM Michal Hocko wrote: > > > > > > On Mon 09-01-23 12:53:23, Suren Baghdasaryan wrote: > > >

Re: [PATCH 17/41] mm/mmap: move VMA locking before anon_vma_lock_write call

On Wed, Jan 18, 2023 at 1:33 PM Michal Hocko wrote: > > On Wed 18-01-23 10:09:29, Suren Baghdasaryan wrote: > > On Wed, Jan 18, 2023 at 1:23 AM Michal Hocko wrote: > > > > > > On Tue 17-01-23 18:01:01, Suren Baghdasaryan wrote: > > > > On Tue, J

Re: [PATCH 39/41] kernel/fork: throttle call_rcu() calls in vm_area_free

On Mon, Jan 23, 2023 at 1:56 AM Michal Hocko wrote: > > On Fri 20-01-23 09:50:01, Suren Baghdasaryan wrote: > > On Fri, Jan 20, 2023 at 9:32 AM Matthew Wilcox wrote: > [...] > > > The page fault handler (or whatever other reader -- ptrace, proc, etc) > > > should

Re: [PATCH 39/41] kernel/fork: throttle call_rcu() calls in vm_area_free

On Mon, Jan 23, 2023 at 8:55 AM Michal Hocko wrote: > > On Mon 23-01-23 08:22:53, Suren Baghdasaryan wrote: > > On Mon, Jan 23, 2023 at 1:56 AM Michal Hocko wrote: > > > > > > On Fri 20-01-23 09:50:01, Suren Baghdasaryan wrote: > > > > On F

Re: [PATCH 39/41] kernel/fork: throttle call_rcu() calls in vm_area_free

On Mon, Jan 23, 2023 at 11:31 AM Matthew Wilcox wrote: > > On Mon, Jan 23, 2023 at 08:18:37PM +0100, Michal Hocko wrote: > > On Mon 23-01-23 18:23:08, Matthew Wilcox wrote: > > > On Mon, Jan 23, 2023 at 09:46:20AM -0800, Suren Baghdasaryan wrote: > > [...] >

Re: [PATCH 39/41] kernel/fork: throttle call_rcu() calls in vm_area_free

On Mon, Jan 23, 2023 at 12:00 PM Michal Hocko wrote: > > On Mon 23-01-23 19:30:43, Matthew Wilcox wrote: > > On Mon, Jan 23, 2023 at 08:18:37PM +0100, Michal Hocko wrote: > > > On Mon 23-01-23 18:23:08, Matthew Wilcox wrote: > > > > On Mon, Jan 23, 2023 at 09:

Re: [PATCH 39/41] kernel/fork: throttle call_rcu() calls in vm_area_free

On Mon, Jan 23, 2023 at 9:16 AM Michal Hocko wrote: > > On Mon 23-01-23 09:07:34, Suren Baghdasaryan wrote: > > On Mon, Jan 23, 2023 at 8:55 AM Michal Hocko wrote: > > > > > > On Mon 23-01-23 08:22:53, Suren Baghdasaryan wrote: > > > > On Mon, J

Re: [PATCH 39/41] kernel/fork: throttle call_rcu() calls in vm_area_free

On Mon, Jan 23, 2023 at 1:59 AM 'Michal Hocko' via kernel-team wrote: > > On Fri 20-01-23 08:20:43, Suren Baghdasaryan wrote: > > On Fri, Jan 20, 2023 at 12:52 AM Michal Hocko wrote: > > > > > > On Thu 19-01-23 10:52:03, Suren Baghdasaryan wrote: > > > &

Re: [PATCH 39/41] kernel/fork: throttle call_rcu() calls in vm_area_free

On Mon, Jan 23, 2023 at 10:23 AM Matthew Wilcox wrote: > > On Mon, Jan 23, 2023 at 09:46:20AM -0800, Suren Baghdasaryan wrote: > > On Mon, Jan 23, 2023 at 9:16 AM Michal Hocko wrote: > > > > > > On Mon 23-01-23 09:07:34, Suren Baghdasaryan wrote: > > > &

Re: [PATCH 39/41] kernel/fork: throttle call_rcu() calls in vm_area_free

On Fri, Jan 20, 2023 at 12:52 AM Michal Hocko wrote: > > On Thu 19-01-23 10:52:03, Suren Baghdasaryan wrote: > > On Thu, Jan 19, 2023 at 4:59 AM Michal Hocko wrote: > > > > > > On Mon 09-01-23 12:53:34, Suren Baghdasaryan wrote: > > > > call_rcu() can

Re: [PATCH 39/41] kernel/fork: throttle call_rcu() calls in vm_area_free

On Fri, Jan 20, 2023 at 8:20 AM Suren Baghdasaryan wrote: > > On Fri, Jan 20, 2023 at 12:52 AM Michal Hocko wrote: > > > > On Thu 19-01-23 10:52:03, Suren Baghdasaryan wrote: > > > On Thu, Jan 19, 2023 at 4:59 AM Michal Hocko wrote: > > > > > > >

Re: [PATCH 39/41] kernel/fork: throttle call_rcu() calls in vm_area_free

On Fri, Jan 20, 2023 at 9:08 AM Liam R. Howlett wrote: > > * Matthew Wilcox [230120 11:50]: > > On Fri, Jan 20, 2023 at 08:45:21AM -0800, Suren Baghdasaryan wrote: > > > On Fri, Jan 20, 2023 at 8:20 AM Suren Baghdasaryan > > > wrote: > > > > >

Re: [PATCH v4 4/7] mm: replace vma->vm_flags direct modifications with modifier calls

On Tue, Jan 31, 2023 at 12:32 AM Hyeonggon Yoo <42.hye...@gmail.com> wrote: > > On Thu, Jan 26, 2023 at 11:37:49AM -0800, Suren Baghdasaryan wrote: > > Replace direct modifications to vma->vm_flags with calls to modifier > > functions to be able to track flag chan

Re: [PATCH] powerpc/mm: fix mmap_lock bad unlock

ry VMA lock-based page fault handling > >> first") > >> Reported-by: Sachin Sant > >> Link: > >> https://lore.kernel.org/linux-mm/842502fb-f99c-417c-9648-a37d0ecdc...@linux.ibm.com > >> Cc: Suren Baghdasaryan > >> Signed-off-by: Laurent Dufour >

Re: [PATCH v4 31/33] powerc/mm: try VMA lock-based page fault handling first

On Mon, Feb 27, 2023 at 9:37 AM Suren Baghdasaryan wrote: > > From: Laurent Dufour > > Attempt VMA lock-based page fault handling first, and fall back to the > existing mmap_lock-based handling if that fails. > Copied from "x86/mm: try VMA lock-based page fault han

Re: [PATCH v4 0/7] introduce vm_flags modifier functions

On Tue, Mar 14, 2023 at 1:11 PM Alex Williamson wrote: > > On Thu, 26 Jan 2023 11:37:45 -0800 > Suren Baghdasaryan wrote: > > > This patchset was originally published as a part of per-VMA locking [1] and > > was split after suggestion that it's viable on its own and to f

Re: [PATCH v4 0/7] introduce vm_flags modifier functions

On Fri, Mar 17, 2023 at 3:41 PM Alex Williamson wrote: > > On Fri, 17 Mar 2023 12:08:32 -0700 > Suren Baghdasaryan wrote: > > > On Tue, Mar 14, 2023 at 1:11 PM Alex Williamson > > wrote: > > > > > > On Thu, 26 Jan 2023 11:37:45 -0800 > > > Sur

Re: [PATCH v3 17/35] mm/mmap: write-lock VMA before shrinking or expanding it

: > > Reviewed-by: Liam R. Howlett > > > > * Suren Baghdasaryan [230216 00:18]: > > > vma_expand and vma_shrink change VMA boundaries. Expansion might also > > > result in freeing of an adjacent VMA. Write-lock affected VMAs to prevent > > > concurrent page fa

Re: [PATCH v3 23/35] mm/mmap: prevent pagefault handler from racing with mmu_notifier registration

On Thu, Feb 23, 2023 at 12:06 PM Liam R. Howlett wrote: > > * Suren Baghdasaryan [230216 00:18]: > > Page fault handlers might need to fire MMU notifications while a new > > notifier is being registered. Modify mm_take_all_locks to write-lock all > > VMAs and prevent t

Re: [PATCH v3 24/35] mm: introduce vma detached flag

nd meets your usecase? > > * Suren Baghdasaryan [230216 00:18]: > > Per-vma locking mechanism will search for VMA under RCU protection and > > then after locking it, has to ensure it was not removed from the VMA > > tree after we found it. To make this check efficient, introduce a &

Re: [PATCH 1/1] mm/nommu: remove unnecessary VMA locking

On Thu, Mar 2, 2023 at 1:41 AM David Hildenbrand wrote: > > On 01.03.23 20:04, Suren Baghdasaryan wrote: > > Since CONFIG_PER_VMA_LOCK depends on CONFIG_MMU, the changes in nommu > > are not needed. Remove them. > > > > Fixes: bad94decd6a4 ("mm: write-lock

Re: [PATCH 1/1] mm/nommu: remove unnecessary VMA locking

On Fri, Mar 3, 2023 at 1:05 AM David Hildenbrand wrote: > > >> > >> Just a general comment: usually, if review of the original series is > >> still going on, it makes a lot more sense to raise such things in the > >> original series so the author can fixup while things are still in > >>

[PATCH v3 04/35] maple_tree: remove extra smp_wmb() from mas_dead_leaves()

From: Liam Howlett The call to mte_set_dead_node() before the smp_wmb() already calls smp_wmb() so this is not needed. This is an optimization for the RCU mode of the maple tree. Fixes: 54a611b60590 ("Maple Tree: add new data structure") Signed-off-by: Liam Howlett Signed-off

[PATCH v3 05/35] maple_tree: Fix write memory barrier of nodes once dead for RCU mode

e RCU mode of the maple tree. Fixes: 54a611b60590 ("Maple Tree: add new data structure") Signed-off-by: Liam R. Howlett Signed-off-by: Suren Baghdasaryan --- lib/maple_tree.c | 7 +-- tools/testing/radix-tree/maple.c | 16 2 files changed, 21 insert

[PATCH v3 18/35] mm/mremap: write-lock VMA while remapping it to a new address range

Write-lock VMA as locked before copying it and when copy_vma produces a new VMA. Signed-off-by: Suren Baghdasaryan Reviewed-by: Laurent Dufour --- mm/mmap.c | 1 + mm/mremap.c | 1 + 2 files changed, 2 insertions(+) diff --git a/mm/mmap.c b/mm/mmap.c index f079e5bbcd57..0eaa3d1a6cd1 100644

[PATCH v3 17/35] mm/mmap: write-lock VMA before shrinking or expanding it

vma_expand and vma_shrink change VMA boundaries. Expansion might also result in freeing of an adjacent VMA. Write-lock affected VMAs to prevent concurrent page faults. Signed-off-by: Suren Baghdasaryan --- mm/mmap.c | 5 + 1 file changed, 5 insertions(+) diff --git a/mm/mmap.c b/mm/mmap.c

[PATCH v3 31/35] x86/mm: try VMA lock-based page fault handling first

Attempt VMA lock-based page fault handling first, and fall back to the existing mmap_lock-based handling if that fails. Signed-off-by: Suren Baghdasaryan --- arch/x86/Kconfig| 1 + arch/x86/mm/fault.c | 36 2 files changed, 37 insertions(+) diff --git

[PATCH v3 30/35] mm: introduce per-VMA lock statistics

Add a new CONFIG_PER_VMA_LOCK_STATS config option to dump extra statistics about handling page fault under VMA lock. Signed-off-by: Suren Baghdasaryan --- include/linux/vm_event_item.h | 6 ++ include/linux/vmstat.h| 6 ++ mm/Kconfig.debug | 6 ++ mm/memory.c

[PATCH v3 26/35] mm: fall back to mmap_lock if vma->anon_vma is not yet set

ed-off-by: Suren Baghdasaryan --- mm/memory.c | 4 1 file changed, 4 insertions(+) diff --git a/mm/memory.c b/mm/memory.c index 5e1c124552a1..13369ff15ec1 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5242,6 +5242,10 @@ struct vm_area_struct *lock_vma_under_rcu(struct mm_stru

[PATCH v3 25/35] mm: introduce lock_vma_under_rcu to be used from arch-specific code

is acquired. VMA lock statistics are updated according to the results. For now only anonymous VMAs can be searched this way. In other cases the function returns NULL. Signed-off-by: Suren Baghdasaryan --- include/linux/mm.h | 3 +++ mm/memory.c| 46

[PATCH v3 35/35] mm: separate vma->lock from vm_area_struct

k anymore. A number of drivers allocate a pseudo VMA on the stack but they never use the VMA's lock, therefore it does not need to be allocated. The future drivers which might need the VMA lock should use vm_area_alloc()/vm_area_free() to allocate the VMA. Signed-off-by: Suren Baghdasaryan --- incl

[PATCH v3 09/35] mm: introduce CONFIG_PER_VMA_LOCK

needs modifications to handle faults under VMA lock. Signed-off-by: Suren Baghdasaryan --- mm/Kconfig | 12 1 file changed, 12 insertions(+) diff --git a/mm/Kconfig b/mm/Kconfig index ca98b2072df5..2e4a7e61768a 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -1211,6 +1211,18 @@ config

[PATCH v3 08/35] mm: Enable maple tree RCU mode by default.

to ensure the nodes remain valid for readers. Signed-off-by: Liam R. Howlett Signed-off-by: Suren Baghdasaryan --- include/linux/mm_types.h | 3 ++- kernel/fork.c| 3 +++ mm/mmap.c| 4 +++- 3 files changed, 8 insertions(+), 2 deletions(-) diff --git a/include/linux/mm

[PATCH v3 27/35] mm: add FAULT_FLAG_VMA_LOCK flag

Add a new flag to distinguish page faults handled under protection of per-vma lock. Signed-off-by: Suren Baghdasaryan Reviewed-by: Laurent Dufour --- include/linux/mm.h | 3 ++- include/linux/mm_types.h | 1 + 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/include/linux

<    1   2   3   4   >