On Wed, Apr 3, 2024 at 12:58 AM Kefeng Wang wrote:
>
>
>
> On 2024/4/3 13:59, Suren Baghdasaryan wrote:
> > On Tue, Apr 2, 2024 at 12:53 AM Kefeng Wang
> > wrote:
> >>
> >> The vm_flags of vma already checked under per-VMA lock, if it is a
> &g
safe to me.
Using (mm != NULL) to indicate that we are holding mmap_lock is not
ideal but I guess that works.
Reviewed-by: Suren Baghdasaryan
> ---
> arch/x86/mm/fault.c | 23 ++-
> 1 file changed, 14 insertions(+), 9 deletions(-)
>
> diff --git a/arch/x86
On Tue, Apr 2, 2024 at 12:53 AM Kefeng Wang wrote:
>
> The vm_flags of vma already checked under per-VMA lock, if it is a
> bad access, directly handle error and return, there is no need to
> lock_mm_and_find_vma() and check vm_flags again.
>
> Signed-off-by: Kefeng Wang
On Tue, Apr 2, 2024 at 12:53 AM Kefeng Wang wrote:
>
> The vm_flags of vma already checked under per-VMA lock, if it is a
> bad access, directly handle error and return, there is no need to
> lock_mm_and_find_vma() and check vm_flags again.
>
> Signed-off-by: Kefeng Wang
Code-wise looks correct
ff-by: Kefeng Wang
Reviewed-by: Suren Baghdasaryan
> ---
> arch/arm/mm/fault.c | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm/mm/fault.c b/arch/arm/mm/fault.c
> index 439dc6a26bb9..5c4b417e24f9 100644
> --- a/arch/arm/mm/fault.c
> ++
On Tue, Apr 2, 2024 at 10:19 PM Suren Baghdasaryan wrote:
>
> On Tue, Apr 2, 2024 at 12:53 AM Kefeng Wang
> wrote:
> >
> > The vm_flags of vma already checked under per-VMA lock, if it is a
> > bad access, directly set fault to VM_FAULT_BADACCESS an
34% in lmbench 'lat_sig -P 1 prot lat_sig'.
The change makes sense to me. Per-VMA lock is enough to keep
vma->vm_flags stable, so no need to retry with mmap_lock.
>
> Signed-off-by: Kefeng Wang
Reviewed-by: Suren Baghdasaryan
> ---
> arch/arm64/mm/fault.c | 4 +++-
> 1
On Tue, Apr 2, 2024 at 12:53 AM Kefeng Wang wrote:
>
> The __do_page_fault() only check vma->flags and call handle_mm_fault(),
> and only called by do_page_fault(), let's squash it into do_page_fault()
> to cleanup code.
>
> Signed-off-by: Kefeng Wang
Reviewed-b
On Thu, Jan 25, 2024 at 6:04 PM Andrew Morton wrote:
>
> On Mon, 22 Jan 2024 22:43:05 -0800 Suren Baghdasaryan
> wrote:
>
> > The change [1] missed ARM architecture when fixing major fault accounting
> > for page fault retry under per-VMA lock. Add missing code to fix AR
On Sun, Jan 21, 2024 at 11:38 PM Suren Baghdasaryan wrote:
>
> On Sat, Jan 20, 2024 at 1:15 PM Russell King (Oracle)
> wrote:
> >
> > On Sat, Jan 20, 2024 at 09:09:47PM +,
> > patchwork-bot+linux-ri...@kernel.org wrote:
> > > Hello:
> > >
&g
a1992 ("mm: handle read faults under the VMA lock")
Reported-by: Russell King (Oracle)
Signed-off-by: Suren Baghdasaryan
---
arch/arm/mm/fault.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/arm/mm/fault.c b/arch/arm/mm/fault.c
index e96fb40b9cc3..07565b593ed6 100644
--- a
On Sat, Jan 20, 2024 at 1:15 PM Russell King (Oracle)
wrote:
>
> On Sat, Jan 20, 2024 at 09:09:47PM +,
> patchwork-bot+linux-ri...@kernel.org wrote:
> > Hello:
> >
> > This patch was applied to riscv/linux.git (fixes)
> > by Andrew Morton :
> >
> > On Tue, 26 Dec 2023 13:46:10 -0800 you
://lore.kernel.org/all/20231006195318.4087158-6-wi...@infradead.org/
Fixes: 12214eba1992 ("mm: handle read faults under the VMA lock")
Cc: Matthew Wilcox
Signed-off-by: Suren Baghdasaryan
---
arch/arm64/mm/fault.c | 2 ++
arch/powerpc/mm/fault.c | 2 ++
arch/riscv/mm/fault.c | 2 ++
ar
On Wed, Aug 9, 2023 at 2:07 PM Mateusz Guzik wrote:
>
> On 8/5/23, Suren Baghdasaryan wrote:
> > On Fri, Aug 4, 2023 at 6:06 PM Mateusz Guzik wrote:
> >>
> >> On 8/5/23, Linus Torvalds wrote:
> >> > On Fri, 4 Aug 2023 at 16:25, Mateusz Guzik wrot
On Fri, Aug 4, 2023 at 6:06 PM Mateusz Guzik wrote:
>
> On 8/5/23, Linus Torvalds wrote:
> > On Fri, 4 Aug 2023 at 16:25, Mateusz Guzik wrote:
> >>
> >> I know of these guys, I think they are excluded as is -- they go
> >> through access_remote_vm, starting with:
> >> if
On Fri, Aug 4, 2023 at 6:17 PM Mateusz Guzik wrote:
>
> On 8/5/23, Suren Baghdasaryan wrote:
> > On Fri, Aug 4, 2023 at 5:49 PM Mateusz Guzik wrote:
> >> However, the other users (that I know of ) go through the mmap
> >> semaphore to mess with anyth
On Fri, Aug 4, 2023 at 5:49 PM Mateusz Guzik wrote:
>
> On 8/5/23, Suren Baghdasaryan wrote:
> > On Fri, Aug 4, 2023 at 5:26 PM Suren Baghdasaryan
> > wrote:
> >>
> >> On Fri, Aug 4, 2023 at 5:15 PM Linus Torvalds
> >> wrote:
> >> >
On Fri, Aug 4, 2023 at 5:26 PM Suren Baghdasaryan wrote:
>
> On Fri, Aug 4, 2023 at 5:15 PM Linus Torvalds
> wrote:
> >
> > On Fri, 4 Aug 2023 at 16:25, Mateusz Guzik wrote:
> > >
> > > I know of these guys, I think they are excluded as is -- they go
&
On Fri, Aug 4, 2023 at 5:15 PM Linus Torvalds
wrote:
>
> On Fri, 4 Aug 2023 at 16:25, Mateusz Guzik wrote:
> >
> > I know of these guys, I think they are excluded as is -- they go
> > through access_remote_vm, starting with:
> > if (mmap_read_lock_killable(mm))
> > return
On Thu, Jul 13, 2023 at 9:15 AM Matthew Wilcox wrote:
>
> > +int try_vma_locked_page_fault(struct vm_locked_fault *vmlf, vm_fault_t
> > *ret)
> > +{
> > + struct vm_area_struct *vma;
> > + vm_fault_t fault;
>
>
> On Thu, Jul 13, 2023 at 05:53:29PM +0800, Kefeng Wang wrote:
> > +#define
> On Mon, Feb 27, 2023 at 09:35:59AM -0800, Suren Baghdasaryan wrote:
> > > >
> > > > <...>
> > > >
> > > >> Laurent Dufour (1):
> > > >> powerc/mm: try VMA lock-based page fault handling first
> > > >
> >
On Sat, Jul 8, 2023 at 4:10 PM Suren Baghdasaryan wrote:
>
> On Sat, Jul 8, 2023 at 4:04 PM Hugh Dickins wrote:
> >
> > Lockdep is certainly right to complain about
> > (>vm_lock->lock){}-{3:3}, at: vma_start_write+0x2d/0x3f
> >
On Sat, Jul 8, 2023 at 4:04 PM Hugh Dickins wrote:
>
> Lockdep is certainly right to complain about
> (>vm_lock->lock){}-{3:3}, at: vma_start_write+0x2d/0x3f
>but task is already holding lock:
> (>i_mmap_rwsem){+.+.}-{3:3}, at: mmap_region+0x4dc/0x6db
> Invert those to the
On Sat, Jul 8, 2023 at 3:54 PM Linus Torvalds
wrote:
>
> On Sat, 8 Jul 2023 at 15:36, Suren Baghdasaryan wrote:
> >
> > On Sat, Jul 8, 2023 at 2:18 PM Linus Torvalds
> > >
> > > Again - maybe I messed up, but it really feels like the missing
> >
On Sat, Jul 8, 2023 at 2:18 PM Linus Torvalds
wrote:
>
> On Sat, 8 Jul 2023 at 12:12, Suren Baghdasaryan wrote:
> >
> > kernel/fork.c | 1 +
> > 1 file changed, 1 insertion(+)
>
> I ended up editing your explanation a lot.
>
> I'm not convinced that the bug
On Sat, Jul 8, 2023 at 12:23 PM Linus Torvalds
wrote:
>
> On Sat, 8 Jul 2023 at 12:17, Suren Baghdasaryan wrote:
> >
> > Do you want me to disable per-VMA locks by default as well?
>
> No. I seriously believe that if the per-vma locking is so broken that
&
On Sat, Jul 8, 2023 at 12:12 PM Suren Baghdasaryan wrote:
>
> When forking a child process, parent write-protects an anonymous page
> and COW-shares it with the child being forked using copy_present_pte().
> Parent's TLB is flushed right before we drop the parent's mmap_lock i
?id=217624
Fixes: 0bff0aaea03e ("x86/mm: try VMA lock-based page fault handling first")
Cc: sta...@vger.kernel.org
Signed-off-by: Suren Baghdasaryan
---
kernel/fork.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/kernel/fork.c b/kernel/fork.c
index b85814e614a5..d2e12b6d2
it into the VMA tree.
Other places where a new VMA is added into VMA tree do not modify it
after the insertion, so do not need the same locking.
Cc: sta...@vger.kernel.org
Signed-off-by: Suren Baghdasaryan
---
mm/mmap.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/mm/mmap.c b/mm/mmap.c
index
With recent changes necessitating mmap_lock to be held for write while
expanding a stack, per-VMA locks should follow the same rules and be
write-locked to prevent page faults into the VMA being expanded. Add
the necessary locking.
Cc: sta...@vger.kernel.org
Signed-off-by: Suren Baghdasaryan
On Sat, Jul 8, 2023 at 11:05 AM Linus Torvalds
wrote:
>
> On Sat, 8 Jul 2023 at 10:39, Andrew Morton wrote:
> >
> > That was the v1 fix, but after some discussion
> > (https://lkml.kernel.org/r/20230705063711.2670599-1-sur...@google.com)
> > it was decided to take the "excessive" approach.
>
>
On Wed, Jul 5, 2023 at 9:14 AM Suren Baghdasaryan wrote:
>
> On Wed, Jul 5, 2023 at 8:49 AM Andrew Morton
> wrote:
> >
> > On Wed, 5 Jul 2023 10:51:57 +0200 "Linux regression tracking (Thorsten
> > Leemhuis)" wrote:
> >
> > > >>
On Wed, Jul 5, 2023 at 8:49 AM Andrew Morton wrote:
>
> On Wed, 5 Jul 2023 10:51:57 +0200 "Linux regression tracking (Thorsten
> Leemhuis)" wrote:
>
> > >>> I'm in wait-a-few-days-mode on this. To see if we have a backportable
> > >>> fix rather than disabling the feature in -stable.
> >
> >
On Tue, Jul 4, 2023 at 3:04 PM Suren Baghdasaryan wrote:
>
> On Tue, Jul 4, 2023 at 2:28 PM Andrew Morton
> wrote:
> >
> > On Tue, 4 Jul 2023 13:22:54 -0700 Suren Baghdasaryan
> > wrote:
> >
> > > On Tue, Jul 4, 2023 at 9:18 AM Andrew Morton
>
On Tue, Jul 4, 2023 at 2:28 PM Andrew Morton wrote:
>
> On Tue, 4 Jul 2023 13:22:54 -0700 Suren Baghdasaryan
> wrote:
>
> > On Tue, Jul 4, 2023 at 9:18 AM Andrew Morton
> > wrote:
> > >
> > > On Tue, 4 Jul 2023 09:00:19 +0100 Greg KH
>
On Tue, Jul 4, 2023 at 9:18 AM Andrew Morton wrote:
>
> On Tue, 4 Jul 2023 09:00:19 +0100 Greg KH wrote:
>
> > > > > > Thanks! I'll investigate this later today. After discussing with
> > > > > > Andrew, we would like to disable CONFIG_PER_VMA_LOCK by default
> > > > > > until
> > > > > > the
On Mon, Jul 3, 2023 at 11:44 AM Greg KH wrote:
>
> On Mon, Jul 03, 2023 at 11:27:19AM -0700, Suren Baghdasaryan wrote:
> > On Mon, Jul 3, 2023 at 11:08 AM Suren Baghdasaryan
> > wrote:
> > >
> > > On Mon, Jul 3, 2023 at 2:53 AM Linux regression track
On Mon, Jul 3, 2023 at 11:08 AM Suren Baghdasaryan wrote:
>
> On Mon, Jul 3, 2023 at 2:53 AM Linux regression tracking (Thorsten
> Leemhuis) wrote:
> >
> > On 02.07.23 14:27, Bagas Sanjaya wrote:
> > > I notice a regression report on Bugzilla [1]. Quoting from it
y memory corruption with either my
> reproducer or the original program.
> ```
>
> FWIW: 0bff0aaea03 ("x86/mm: try VMA lock-based page fault handling
> first") [merged for v6.4-rc1, authored by Suren Baghdasaryan [already CCed]]
>
> That's the same commit that
On Fri, Jun 30, 2023 at 1:43 AM Jiri Slaby wrote:
>
> On 30. 06. 23, 10:28, Jiri Slaby wrote:
> > > 2348
> > clone3({flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID,
> > child_tid=0x7fcaa5882990,
On Thu, Jun 29, 2023 at 7:40 AM Jiri Slaby wrote:
>
> Hi,
>
> On 27. 02. 23, 18:36, Suren Baghdasaryan wrote:
> > Attempt VMA lock-based page fault handling first, and fall back to the
> > existing mmap_lock-based handling if that fails.
> >
> &g
On Fri, Mar 17, 2023 at 3:41 PM Alex Williamson
wrote:
>
> On Fri, 17 Mar 2023 12:08:32 -0700
> Suren Baghdasaryan wrote:
>
> > On Tue, Mar 14, 2023 at 1:11 PM Alex Williamson
> > wrote:
> > >
> > > On Thu, 26 Jan 2023 11:37:45 -0800
> > > Sur
On Tue, Mar 14, 2023 at 1:11 PM Alex Williamson
wrote:
>
> On Thu, 26 Jan 2023 11:37:45 -0800
> Suren Baghdasaryan wrote:
>
> > This patchset was originally published as a part of per-VMA locking [1] and
> > was split after suggestion that it's viable on its own and to f
On Mon, Feb 27, 2023 at 9:37 AM Suren Baghdasaryan wrote:
>
> From: Laurent Dufour
>
> Attempt VMA lock-based page fault handling first, and fall back to the
> existing mmap_lock-based handling if that fails.
> Copied from "x86/mm: try VMA lock-based page fault han
ry VMA lock-based page fault handling
> >> first")
> >> Reported-by: Sachin Sant
> >> Link:
> >> https://lore.kernel.org/linux-mm/842502fb-f99c-417c-9648-a37d0ecdc...@linux.ibm.com
> >> Cc: Suren Baghdasaryan
> >> Signed-off-by: Laurent Dufour
>
On Fri, Mar 3, 2023 at 1:05 AM David Hildenbrand wrote:
>
> >>
> >> Just a general comment: usually, if review of the original series is
> >> still going on, it makes a lot more sense to raise such things in the
> >> original series so the author can fixup while things are still in
> >>
On Thu, Mar 2, 2023 at 1:41 AM David Hildenbrand wrote:
>
> On 01.03.23 20:04, Suren Baghdasaryan wrote:
> > Since CONFIG_PER_VMA_LOCK depends on CONFIG_MMU, the changes in nommu
> > are not needed. Remove them.
> >
> > Fixes: bad94decd6a4 ("mm: write-lock
On Wed, Mar 1, 2023 at 4:54 PM Hyeonggon Yoo <42.hye...@gmail.com> wrote:
>
> On Wed, Mar 01, 2023 at 10:42:48AM -0800, Suren Baghdasaryan wrote:
> > On Wed, Mar 1, 2023 at 10:34 AM Suren Baghdasaryan
> > wrote:
> > >
> > > On Tue, Feb 28, 2023 at 1
On Wed, Mar 1, 2023 at 10:34 AM Suren Baghdasaryan wrote:
>
> On Tue, Feb 28, 2023 at 11:57 PM Hyeonggon Yoo <42.hye...@gmail.com> wrote:
> >
> > On Wed, Mar 01, 2023 at 07:43:33AM +, Hyeonggon Yoo wrote:
> > > On Mon, Feb 27, 2023 at 09:36:17AM -0800, Suren
ocalhost/
Signed-off-by: Suren Baghdasaryan
---
Fix cleanly applies over mm-unstable, SHA in "Fixes" is from that tree.
mm/nommu.c | 5 -
1 file changed, 5 deletions(-)
diff --git a/mm/nommu.c b/mm/nommu.c
index 2ab162d773e2..57ba243c6a37 100644
--- a/mm/nommu.c
+++ b/mm/nommu.c
On Wed, Mar 1, 2023 at 10:34 AM Suren Baghdasaryan wrote:
>
> On Tue, Feb 28, 2023 at 11:57 PM Hyeonggon Yoo <42.hye...@gmail.com> wrote:
> >
> > On Wed, Mar 01, 2023 at 07:43:33AM +, Hyeonggon Yoo wrote:
> > > On Mon, Feb 27, 2023 at 09:36:17AM -0800, Suren
On Tue, Feb 28, 2023 at 11:57 PM Hyeonggon Yoo <42.hye...@gmail.com> wrote:
>
> On Wed, Mar 01, 2023 at 07:43:33AM +, Hyeonggon Yoo wrote:
> > On Mon, Feb 27, 2023 at 09:36:17AM -0800, Suren Baghdasaryan wrote:
> > > Write-locking VMAs before isolating t
On Tue, Feb 28, 2023 at 4:06 AM Punit Agrawal
wrote:
>
> Punit Agrawal writes:
>
> > Suren Baghdasaryan writes:
> >
> >> Previous version:
> >> v1: https://lore.kernel.org/all/20230109205336.3665937-1-sur...@google.com/
> >> RFC: https:/
k anymore. A number of drivers allocate a pseudo
VMA on the stack but they never use the VMA's lock, therefore it does
not need to be allocated. The future drivers which might need the VMA
lock should use vm_area_alloc()/vm_area_free() to allocate the VMA.
Signed-off-by: Suren Baghdasaryan
---
incl
caused by it.
Signed-off-by: Suren Baghdasaryan
---
include/linux/mm.h | 2 ++
kernel/fork.c | 2 +-
mm/mmap.c | 11 +++
3 files changed, 10 insertions(+), 5 deletions(-)
diff --git a/include/linux/mm.h b/include/linux/mm.h
index d07ac92f..5e142bfe7a58 100644
From: Laurent Dufour
Attempt VMA lock-based page fault handling first, and fall back to the
existing mmap_lock-based handling if that fails.
Copied from "x86/mm: try VMA lock-based page fault handling first"
Signed-off-by: Laurent Dufour
Signed-off-by: Suren Baghdasaryan
---
arch/
Attempt VMA lock-based page fault handling first, and fall back to the
existing mmap_lock-based handling if that fails.
Signed-off-by: Suren Baghdasaryan
---
arch/arm64/Kconfig| 1 +
arch/arm64/mm/fault.c | 36
2 files changed, 37 insertions(+)
diff
Attempt VMA lock-based page fault handling first, and fall back to the
existing mmap_lock-based handling if that fails.
Signed-off-by: Suren Baghdasaryan
---
arch/x86/Kconfig| 1 +
arch/x86/mm/fault.c | 36
2 files changed, 37 insertions(+)
diff --git
Add a new CONFIG_PER_VMA_LOCK_STATS config option to dump extra
statistics about handling page fault under VMA lock.
Signed-off-by: Suren Baghdasaryan
---
include/linux/vm_event_item.h | 6 ++
include/linux/vmstat.h| 6 ++
mm/Kconfig.debug | 6 ++
mm/memory.c
Due to the possibility of handle_userfault dropping mmap_lock, avoid fault
handling under VMA lock and retry holding mmap_lock. This can be handled
more gracefully in the future.
Signed-off-by: Suren Baghdasaryan
Suggested-by: Peter Xu
---
mm/memory.c | 9 +
1 file changed, 9
Due to the possibility of do_swap_page dropping mmap_lock, abort fault
handling under VMA lock and retry holding mmap_lock. This can be handled
more gracefully in the future.
Signed-off-by: Suren Baghdasaryan
Reviewed-by: Laurent Dufour
---
mm/memory.c | 5 +
1 file changed, 5 insertions
ed-off-by: Suren Baghdasaryan
---
mm/memory.c | 4
1 file changed, 4 insertions(+)
diff --git a/mm/memory.c b/mm/memory.c
index bda4c1a991f0..8855846a361b 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -5243,6 +5243,10 @@ struct vm_area_struct *lock_vma_under_rcu(struct
mm_stru
Add a new flag to distinguish page faults handled under protection of
per-vma lock.
Signed-off-by: Suren Baghdasaryan
Reviewed-by: Laurent Dufour
---
include/linux/mm.h | 3 ++-
include/linux/mm_types.h | 1 +
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/include/linux
is acquired. VMA lock statistics are updated according to
the results.
For now only anonymous VMAs can be searched this way. In other cases the
function returns NULL.
Signed-off-by: Suren Baghdasaryan
---
include/linux/mm.h | 3 +++
mm/memory.c| 46
Per-vma locking mechanism will search for VMA under RCU protection and
then after locking it, has to ensure it was not removed from the VMA
tree after we found it. To make this check efficient, introduce a
vma->detached flag to mark VMAs which were removed from the VMA tree.
Signed-off-by: Su
order as in page fault handlers.
Signed-off-by: Suren Baghdasaryan
---
mm/mmap.c | 9 +
1 file changed, 9 insertions(+)
diff --git a/mm/mmap.c b/mm/mmap.c
index ec745586785c..b947d82e8522 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -3486,6 +3486,7 @@ static void vm_lock_mapping(struct
Assert there are no holders of VMA lock for reading when it is about to be
destroyed.
Signed-off-by: Suren Baghdasaryan
---
kernel/fork.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/kernel/fork.c b/kernel/fork.c
index e1dd79c7738c..bdb55f25895d 100644
--- a/kernel/fork.c
+++ b/kernel
to free_pgtables for such scenario.
Signed-off-by: Suren Baghdasaryan
---
mm/internal.h | 2 +-
mm/memory.c | 6 +-
mm/mmap.c | 5 +++--
3 files changed, 9 insertions(+), 4 deletions(-)
diff --git a/mm/internal.h b/mm/internal.h
index 08ce56dbb1d9..fce94775819c 100644
--- a/mm/internal.h
+++ b/mm
Write-locking VMAs before isolating them ensures that page fault
handlers don't operate on isolated VMAs.
Signed-off-by: Suren Baghdasaryan
---
mm/mmap.c | 1 +
mm/nommu.c | 5 +
2 files changed, 6 insertions(+)
diff --git a/mm/mmap.c b/mm/mmap.c
index 1f42b9a52b9b..f7ed357056c4 100644
Write-lock VMA as locked before copying it and when copy_vma produces
a new VMA.
Signed-off-by: Suren Baghdasaryan
Reviewed-by: Laurent Dufour
---
mm/mmap.c | 1 +
mm/mremap.c | 1 +
2 files changed, 2 insertions(+)
diff --git a/mm/mmap.c b/mm/mmap.c
index e73fbb84ce12..1f42b9a52b9b 100644
Write-lock all VMAs which might be affected by a merge, split, expand
or shrink operations. All these operations use vma_prepare() before
making the modifications, therefore it provides a centralized place to
perform VMA locking.
Signed-off-by: Suren Baghdasaryan
---
mm/mmap.c | 10
a fourth lock under which
page tables can be traversed, and so khugepaged must also lock out that
one.
Signed-off-by: Suren Baghdasaryan
---
mm/khugepaged.c | 5 +
mm/rmap.c | 31 ---
2 files changed, 21 insertions(+), 15 deletions(-)
diff --git a/mm
vma_prepare() acquires all locks required before VMA modifications.
Move vma_prepare() before vma_adjust_trans_huge() so that VMA is locked
before any modification.
Signed-off-by: Suren Baghdasaryan
---
mm/mmap.c | 10 +-
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/mm
Updates to vm_flags have to be done with VMA marked as being written for
preventing concurrent page faults or other modifications.
Signed-off-by: Suren Baghdasaryan
---
include/linux/mm.h | 10 +-
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/include/linux/mm.h b/include
ive mmap_lock is released by incrementing mm
sequence counter (mm_lock_seq).
- write downgrade - if the mmap_lock is downgraded to the read lock, all
vma write locks are released as well (effectivelly same as write
unlock).
Signed-off-by: Suren Baghdasaryan
---
include/linux/m
Move mmap_lock assert function definitions up so that they can be used
by other mmap_lock routines.
Signed-off-by: Suren Baghdasaryan
---
include/linux/mmap_lock.h | 24
1 file changed, 12 insertions(+), 12 deletions(-)
diff --git a/include/linux/mmap_lock.h b/include
From: Michel Lespinasse
This prepares for page faults handling under VMA lock, looking up VMAs
under protection of an rcu read lock, instead of the usual mmap read lock.
Signed-off-by: Michel Lespinasse
Signed-off-by: Suren Baghdasaryan
---
include/linux/mm_types.h | 13 ++---
kernel
needs modifications to
handle faults under VMA lock.
Signed-off-by: Suren Baghdasaryan
---
mm/Kconfig | 12
1 file changed, 12 insertions(+)
diff --git a/mm/Kconfig b/mm/Kconfig
index ca98b2072df5..2e4a7e61768a 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -1211,6 +1211,18 @@ config
to ensure the
nodes remain valid for readers.
Signed-off-by: Liam R. Howlett
Signed-off-by: Suren Baghdasaryan
---
include/linux/mm_types.h | 3 ++-
kernel/fork.c| 3 +++
mm/mmap.c| 4 +++-
3 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/include/linux/mm
and parent pointers.
Fixes: 54a611b60590 ("Maple Tree: add new data structure")
Reported-by: Suren Baghdasaryan
Signed-off-by: Liam R. Howlett
---
lib/maple_tree.c | 188 ---
1 file changed, 96 insertions(+), 92 deletions(-)
diff --git a/li
re")
Signed-off-by: Liam R. Howlett
Signed-off-by: Suren Baghdasaryan
---
lib/maple_tree.c | 8 ++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/lib/maple_tree.c b/lib/maple_tree.c
index 6b6eddadd9d2..8ad2d1669fad 100644
--- a/lib/maple_tree.c
+++ b/lib/maple_tree.c
@@ -539
e RCU mode of the maple tree.
Fixes: 54a611b60590 ("Maple Tree: add new data structure")
Signed-off-by: Liam R. Howlett
Signed-off-by: Suren Baghdasaryan
---
lib/maple_tree.c | 7 +--
tools/testing/radix-tree/maple.c | 16
2 files changed, 21 insert
From: Liam Howlett
The call to mte_set_dead_node() before the smp_wmb() already calls
smp_wmb() so this is not needed. This is an optimization for the RCU
mode of the maple tree.
Fixes: 54a611b60590 ("Maple Tree: add new data structure")
Signed-off-by: Liam Howlett
Signed-off
Tree: add new data structure")
Signed-off-by: Liam Howlett
Signed-off-by: Suren Baghdasaryan
---
lib/maple_tree.c | 73
1 file changed, 62 insertions(+), 11 deletions(-)
diff --git a/lib/maple_tree.c b/lib/maple_tree.c
index 089cd76ec379..44
t;)
Signed-off-by: Liam Howlett
Signed-off-by: Suren Baghdasaryan
---
lib/maple_tree.c | 4
1 file changed, 4 insertions(+)
diff --git a/lib/maple_tree.c b/lib/maple_tree.c
index cc356b8369ad..089cd76ec379 100644
--- a/lib/maple_tree.c
+++ b/lib/maple_tree.c
@@ -1360,12 +1360,16 @@ sta
am Howlett
Signed-off-by: Suren Baghdasaryan
---
lib/maple_tree.c | 52 +++-
1 file changed, 43 insertions(+), 9 deletions(-)
diff --git a/lib/maple_tree.c b/lib/maple_tree.c
index 646297cae5d1..cc356b8369ad 100644
--- a/lib/maple_tree.c
+++ b/lib/ma
maple_tree: Fix write memory barrier of nodes once dead for RCU mode
maple_tree: Add smp_rmb() to dead node detection
maple_tree: Add RCU lock checking to rcu callback functions
mm: Enable maple tree RCU mode by default.
Michel Lespinasse (1):
mm: rcu safe VMA freeing
Suren Baghdasaryan (23):
On Fri, Feb 24, 2023 at 8:19 AM Suren Baghdasaryan wrote:
>
> On Fri, Feb 24, 2023 at 8:14 AM Liam R. Howlett
> wrote:
> >
> > * Suren Baghdasaryan [230223 21:06]:
> > > On Thu, Feb 23, 2023 at 5:46 PM Liam R. Howlett
> > > wrote:
> > &g
On Mon, Feb 27, 2023 at 9:19 AM Davidlohr Bueso wrote:
>
> On Fri, 24 Feb 2023, freak07 wrote:
>
> >Here are some measurements from a Pixel 7 Pro that´s running a kernel either
> >with the Per-VMA locks patchset or without.
> >If there´s interest I can provide results of other specific apps as
On Fri, Feb 24, 2023 at 8:14 AM Liam R. Howlett wrote:
>
> * Suren Baghdasaryan [230223 21:06]:
> > On Thu, Feb 23, 2023 at 5:46 PM Liam R. Howlett
> > wrote:
> > >
> > > * Suren Baghdasaryan [230223 16:16]:
> > > > On Thu, Feb 23,
On Thu, Feb 23, 2023 at 5:46 PM Liam R. Howlett wrote:
>
> * Suren Baghdasaryan [230223 16:16]:
> > On Thu, Feb 23, 2023 at 12:28 PM Liam R. Howlett
> > wrote:
> > >
> > >
> > > Wait, I figured a better place to do this.
> > >
> >
:
> > Reviewed-by: Liam R. Howlett
> >
> > * Suren Baghdasaryan [230216 00:18]:
> > > vma_expand and vma_shrink change VMA boundaries. Expansion might also
> > > result in freeing of an adjacent VMA. Write-lock affected VMAs to prevent
> > > concurrent page fa
nd meets your usecase?
>
> * Suren Baghdasaryan [230216 00:18]:
> > Per-vma locking mechanism will search for VMA under RCU protection and
> > then after locking it, has to ensure it was not removed from the VMA
> > tree after we found it. To make this check efficient, introduce a
&
On Thu, Feb 23, 2023 at 12:06 PM Liam R. Howlett
wrote:
>
> * Suren Baghdasaryan [230216 00:18]:
> > Page fault handlers might need to fire MMU notifications while a new
> > notifier is being registered. Modify mm_take_all_locks to write-lock all
> > VMAs and prevent t
On Fri, Feb 17, 2023 at 2:21 AM Hyeonggon Yoo <42.hye...@gmail.com> wrote:
>
> On Fri, Feb 17, 2023 at 11:15 AM Suren Baghdasaryan wrote:
> >
> > On Thu, Feb 16, 2023 at 11:43 AM Suren Baghdasaryan
> > wrote:
> > >
> > > On Thu, Feb
On Fri, Feb 17, 2023 at 8:05 AM Matthew Wilcox wrote:
>
> On Thu, Feb 16, 2023 at 06:14:59PM -0800, Suren Baghdasaryan wrote:
> > On Thu, Feb 16, 2023 at 11:43 AM Suren Baghdasaryan
> > wrote:
> > >
> > > On Thu, Feb 16, 2023 at 7:44 AM Matthew Wilcox
>
On Fri, Feb 17, 2023 at 6:51 AM Liam R. Howlett wrote:
>
> * Suren Baghdasaryan [230216 14:36]:
> > On Thu, Feb 16, 2023 at 7:34 AM Liam R. Howlett
> > wrote:
> > >
> > >
> > > First, sorry I didn't see this before v3..
>
On Thu, Feb 16, 2023 at 11:43 AM Suren Baghdasaryan wrote:
>
> On Thu, Feb 16, 2023 at 7:44 AM Matthew Wilcox wrote:
> >
> > On Wed, Feb 15, 2023 at 09:17:41PM -0800, Suren Baghdasaryan wrote:
> > > When vma->anon_vma is not set, page fault handler will set it by
On Thu, Feb 16, 2023 at 7:44 AM Matthew Wilcox wrote:
>
> On Wed, Feb 15, 2023 at 09:17:41PM -0800, Suren Baghdasaryan wrote:
> > When vma->anon_vma is not set, page fault handler will set it by either
> > reusing anon_vma of an adjacent VMA if VMAs are compatible or by
>
On Thu, Feb 16, 2023 at 7:34 AM Liam R. Howlett wrote:
>
>
> First, sorry I didn't see this before v3..
Feedback at any time is highly appreciated!
>
> * Suren Baghdasaryan [230216 00:18]:
> > While unmapping VMAs, adjacent VMAs might be able to grow into the ar
1 - 100 of 321 matches
Mail list logo
