CVS commit: [netbsd-9] src/sys/uvm
Module Name:src
Committed By: martin
Date: Tue Aug 15 09:46:23 UTC 2023
Modified Files:
src/sys/uvm [netbsd-9]: uvm_fault.c
Log Message:
Pull up following revision(s) (requested by chs in ticket #1714):
sys/uvm/uvm_fault.c: revision 1.234
uvm: prevent TLB invalidation races during COW resolution
When a thread takes a page fault which results in COW resolution,
other threads in the same process can be concurrently accessing that
same mapping on other CPUs. When the faulting thread updates the pmap
entry at the end of COW processing, the resulting TLB invalidations to
other CPUs are not done atomically, so another thread can write to the
new writable page and then a third thread might still read from the
old read-only page, resulting in inconsistent views of the page by the
latter two threads. Fix this by removing the pmap entry entirely for
the original page before we install the new pmap entry for the new
page, so that the new page can only be modified after the old page is
no longer accessible.
This fixes PR 56535 as well as the netbsd versions of problems
described in various bug trackers:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225584
https://reviews.freebsd.org/D14347
https://github.com/golang/go/issues/34988
To generate a diff of this commit:
cvs rdiff -u -r1.206.2.2 -r1.206.2.3 src/sys/uvm/uvm_fault.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
Modified files:
Index: src/sys/uvm/uvm_fault.c
diff -u src/sys/uvm/uvm_fault.c:1.206.2.2 src/sys/uvm/uvm_fault.c:1.206.2.3
--- src/sys/uvm/uvm_fault.c:1.206.2.2 Sun Mar 8 11:01:22 2020
+++ src/sys/uvm/uvm_fault.c Tue Aug 15 09:46:23 2023
@@ -1,4 +1,4 @@
-/* $NetBSD: uvm_fault.c,v 1.206.2.2 2020/03/08 11:01:22 martin Exp $ */
+/* $NetBSD: uvm_fault.c,v 1.206.2.3 2023/08/15 09:46:23 martin Exp $ */
/*
* Copyright (c) 1997 Charles D. Cranor and Washington University.
@@ -32,7 +32,7 @@
*/
#include
-__KERNEL_RCSID(0, "$NetBSD: uvm_fault.c,v 1.206.2.2 2020/03/08 11:01:22 martin Exp $");
+__KERNEL_RCSID(0, "$NetBSD: uvm_fault.c,v 1.206.2.3 2023/08/15 09:46:23 martin Exp $");
#include "opt_uvmhist.h"
@@ -639,8 +639,17 @@ uvmfault_promote(struct uvm_faultinfo *u
goto done;
}
- /* copy page [pg now dirty] */
+ /*
+ * copy the page [pg now dirty]
+ *
+ * Remove the pmap entry now for the old page at this address
+ * so that no thread can modify the new page while any thread
+ * might still see the old page.
+ */
if (opg) {
+ pmap_remove(vm_map_pmap(ufi->orig_map), ufi->orig_rvaddr,
+ ufi->orig_rvaddr + PAGE_SIZE);
+ pmap_update(vm_map_pmap(ufi->orig_map));
uvm_pagecopy(opg, pg);
}
CVS commit: [netbsd-9] src/sys/uvm
Module Name:src Committed By: martin Date: Tue Aug 15 09:46:23 UTC 2023 Modified Files: src/sys/uvm [netbsd-9]: uvm_fault.c Log Message: Pull up following revision(s) (requested by chs in ticket #1714): sys/uvm/uvm_fault.c: revision 1.234 uvm: prevent TLB invalidation races during COW resolution When a thread takes a page fault which results in COW resolution, other threads in the same process can be concurrently accessing that same mapping on other CPUs. When the faulting thread updates the pmap entry at the end of COW processing, the resulting TLB invalidations to other CPUs are not done atomically, so another thread can write to the new writable page and then a third thread might still read from the old read-only page, resulting in inconsistent views of the page by the latter two threads. Fix this by removing the pmap entry entirely for the original page before we install the new pmap entry for the new page, so that the new page can only be modified after the old page is no longer accessible. This fixes PR 56535 as well as the netbsd versions of problems described in various bug trackers: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225584 https://reviews.freebsd.org/D14347 https://github.com/golang/go/issues/34988 To generate a diff of this commit: cvs rdiff -u -r1.206.2.2 -r1.206.2.3 src/sys/uvm/uvm_fault.c Please note that diffs are not public domain; they are subject to the copyright notices on the relevant files.
CVS commit: [netbsd-9] src/sys/uvm
Module Name:src
Committed By: martin
Date: Sat Apr 1 16:22:15 UTC 2023
Modified Files:
src/sys/uvm [netbsd-9]: uvm_map.c
Log Message:
Pull up following revision(s) (requested by riastradh in ticket #1625):
sys/uvm/uvm_map.c: revision 1.403
mmap(2): Avoid arithmetic overflow in search for free space.
PR kern/56900
To generate a diff of this commit:
cvs rdiff -u -r1.362.2.4 -r1.362.2.5 src/sys/uvm/uvm_map.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
Modified files:
Index: src/sys/uvm/uvm_map.c
diff -u src/sys/uvm/uvm_map.c:1.362.2.4 src/sys/uvm/uvm_map.c:1.362.2.5
--- src/sys/uvm/uvm_map.c:1.362.2.4 Sat Apr 1 16:03:48 2023
+++ src/sys/uvm/uvm_map.c Sat Apr 1 16:22:14 2023
@@ -1,4 +1,4 @@
-/* $NetBSD: uvm_map.c,v 1.362.2.4 2023/04/01 16:03:48 martin Exp $ */
+/* $NetBSD: uvm_map.c,v 1.362.2.5 2023/04/01 16:22:14 martin Exp $ */
/*
* Copyright (c) 1997 Charles D. Cranor and Washington University.
@@ -66,7 +66,7 @@
*/
#include
-__KERNEL_RCSID(0, "$NetBSD: uvm_map.c,v 1.362.2.4 2023/04/01 16:03:48 martin Exp $");
+__KERNEL_RCSID(0, "$NetBSD: uvm_map.c,v 1.362.2.5 2023/04/01 16:22:14 martin Exp $");
#include "opt_ddb.h"
#include "opt_pax.h"
@@ -2026,7 +2026,21 @@ uvm_map_findspace(struct vm_map *map, va
/* Try to find the space in the red-black tree */
/* Check slot before any entry */
- hint = topdown ? entry->next->start - length : entry->end;
+ if (topdown) {
+ KASSERTMSG(entry->next->start >= vm_map_min(map),
+ "map=%p entry=%p entry->next=%p"
+ " entry->next->start=0x%"PRIxVADDR" min=0x%"PRIxVADDR,
+ map, entry, entry->next,
+ entry->next->start, vm_map_min(map));
+ if (length > entry->next->start - vm_map_min(map))
+ hint = vm_map_min(map); /* XXX goto wraparound? */
+ else
+ hint = entry->next->start - length;
+ KASSERT(hint >= vm_map_min(map));
+ } else {
+ hint = entry->end;
+ }
+
switch (uvm_map_space_avail(, length, uoffset, align, flags,
topdown, entry)) {
case 1:
CVS commit: [netbsd-9] src/sys/uvm
Module Name:src Committed By: martin Date: Sat Apr 1 16:22:15 UTC 2023 Modified Files: src/sys/uvm [netbsd-9]: uvm_map.c Log Message: Pull up following revision(s) (requested by riastradh in ticket #1625): sys/uvm/uvm_map.c: revision 1.403 mmap(2): Avoid arithmetic overflow in search for free space. PR kern/56900 To generate a diff of this commit: cvs rdiff -u -r1.362.2.4 -r1.362.2.5 src/sys/uvm/uvm_map.c Please note that diffs are not public domain; they are subject to the copyright notices on the relevant files.
CVS commit: [netbsd-9] src/sys/uvm
Module Name:src
Committed By: martin
Date: Sat Apr 1 16:03:48 UTC 2023
Modified Files:
src/sys/uvm [netbsd-9]: uvm_map.c
Log Message:
Pull up following revision(s) (requested by riastradh in ticket #1623):
sys/uvm/uvm_map.c: revision 1.396
uvm(9): Fix mmap optimization for topdown case.
PR kern/51393
To generate a diff of this commit:
cvs rdiff -u -r1.362.2.3 -r1.362.2.4 src/sys/uvm/uvm_map.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
Modified files:
Index: src/sys/uvm/uvm_map.c
diff -u src/sys/uvm/uvm_map.c:1.362.2.3 src/sys/uvm/uvm_map.c:1.362.2.4
--- src/sys/uvm/uvm_map.c:1.362.2.3 Sat Apr 1 16:00:28 2023
+++ src/sys/uvm/uvm_map.c Sat Apr 1 16:03:48 2023
@@ -1,4 +1,4 @@
-/* $NetBSD: uvm_map.c,v 1.362.2.3 2023/04/01 16:00:28 martin Exp $ */
+/* $NetBSD: uvm_map.c,v 1.362.2.4 2023/04/01 16:03:48 martin Exp $ */
/*
* Copyright (c) 1997 Charles D. Cranor and Washington University.
@@ -66,7 +66,7 @@
*/
#include
-__KERNEL_RCSID(0, "$NetBSD: uvm_map.c,v 1.362.2.3 2023/04/01 16:00:28 martin Exp $");
+__KERNEL_RCSID(0, "$NetBSD: uvm_map.c,v 1.362.2.4 2023/04/01 16:03:48 martin Exp $");
#include "opt_ddb.h"
#include "opt_pax.h"
@@ -1940,7 +1940,36 @@ uvm_map_findspace(struct vm_map *map, va
* it, there would be four cases).
*/
- if ((flags & UVM_FLAG_FIXED) == 0 && hint == vm_map_min(map)) {
+ if ((flags & UVM_FLAG_FIXED) == 0 &&
+ hint == (topdown ? vm_map_max(map) : vm_map_min(map))) {
+ /*
+ * The uvm_map_findspace algorithm is monotonic -- for
+ * topdown VM it starts with a high hint and returns a
+ * lower free address; for !topdown VM it starts with a
+ * low hint and returns a higher free address. As an
+ * optimization, start with the first (highest for
+ * topdown, lowest for !topdown) free address.
+ *
+ * XXX This `optimization' probably doesn't actually do
+ * much in practice unless userland explicitly passes
+ * the VM map's minimum or maximum address, which
+ * varies from machine to machine (VM_MAX/MIN_ADDRESS,
+ * e.g. 0x7fbfdfeff000 on amd64 but 0xf000 on
+ * aarch64) and may vary according to other factors
+ * like sysctl vm.user_va0_disable. In particular, if
+ * the user specifies 0 as a hint to mmap, then mmap
+ * will choose a default address which is usually _not_
+ * VM_MAX/MIN_ADDRESS but something else instead like
+ * VM_MAX_ADDRESS - stack size - guard page overhead,
+ * in which case this branch is never hit.
+ *
+ * In fact, this branch appears to have been broken for
+ * two decades between when topdown was introduced in
+ * ~2003 and when it was adapted to handle the topdown
+ * case without violating the monotonicity assertion in
+ * 2022. Maybe Someone^TM should either ditch the
+ * optimization or find a better way to do it.
+ */
entry = map->first_free;
} else {
if (uvm_map_lookup_entry(map, hint, )) {
CVS commit: [netbsd-9] src/sys/uvm
Module Name:src Committed By: martin Date: Sat Apr 1 16:03:48 UTC 2023 Modified Files: src/sys/uvm [netbsd-9]: uvm_map.c Log Message: Pull up following revision(s) (requested by riastradh in ticket #1623): sys/uvm/uvm_map.c: revision 1.396 uvm(9): Fix mmap optimization for topdown case. PR kern/51393 To generate a diff of this commit: cvs rdiff -u -r1.362.2.3 -r1.362.2.4 src/sys/uvm/uvm_map.c Please note that diffs are not public domain; they are subject to the copyright notices on the relevant files.
CVS commit: [netbsd-9] src/sys/uvm
Module Name:src
Committed By: martin
Date: Sat Apr 1 16:00:29 UTC 2023
Modified Files:
src/sys/uvm [netbsd-9]: uvm_map.c
Log Message:
Pull up following revision(s) (requested by riastradh in ticket #1622):
sys/uvm/uvm_map.c: revision 1.395
uvm(9): Fix 19-year-old bug in assertion about mmap hint.
Previously this would _first_ remember the original hint, and _then_
clamp the hint to the VM map's range:
orig_hint = hint;
if (hint < vm_map_min(map)) { /* check ranges ... */
if (flags & UVM_FLAG_FIXED) {
UVMHIST_LOG(maphist,"<- VA below map range",0,0,0,0);
return (NULL);
}
hint = vm_map_min(map);
...
KASSERTMSG(!topdown || hint <= orig_hint, "hint: %#jx, orig_hint: %#jx",
(uintmax_t)hint, (uintmax_t)orig_hint);
Even if nothing else happens in the ellipsis, taking the branch
guarantees the assertion will fail in the topdown case.
To generate a diff of this commit:
cvs rdiff -u -r1.362.2.2 -r1.362.2.3 src/sys/uvm/uvm_map.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
Modified files:
Index: src/sys/uvm/uvm_map.c
diff -u src/sys/uvm/uvm_map.c:1.362.2.2 src/sys/uvm/uvm_map.c:1.362.2.3
--- src/sys/uvm/uvm_map.c:1.362.2.2 Fri Nov 1 18:24:31 2019
+++ src/sys/uvm/uvm_map.c Sat Apr 1 16:00:28 2023
@@ -1,4 +1,4 @@
-/* $NetBSD: uvm_map.c,v 1.362.2.2 2019/11/01 18:24:31 martin Exp $ */
+/* $NetBSD: uvm_map.c,v 1.362.2.3 2023/04/01 16:00:28 martin Exp $ */
/*
* Copyright (c) 1997 Charles D. Cranor and Washington University.
@@ -66,7 +66,7 @@
*/
#include
-__KERNEL_RCSID(0, "$NetBSD: uvm_map.c,v 1.362.2.2 2019/11/01 18:24:31 martin Exp $");
+__KERNEL_RCSID(0, "$NetBSD: uvm_map.c,v 1.362.2.3 2023/04/01 16:00:28 martin Exp $");
#include "opt_ddb.h"
#include "opt_pax.h"
@@ -1882,12 +1882,17 @@ uvm_map_findspace(struct vm_map *map, va
uvm_map_check(map, "map_findspace entry");
/*
- * remember the original hint. if we are aligning, then we
- * may have to try again with no alignment constraint if
- * we fail the first time.
+ * Clamp the hint to the VM map's min/max address, and remmeber
+ * the clamped original hint. Remember the original hint,
+ * clamped to the min/max address. If we are aligning, then we
+ * may have to try again with no alignment constraint if we
+ * fail the first time.
+ *
+ * We use the original hint to verify later that the search has
+ * been monotonic -- that is, nonincreasing or nondecreasing,
+ * according to topdown or !topdown respectively. But the
+ * clamping is not monotonic.
*/
-
- orig_hint = hint;
if (hint < vm_map_min(map)) { /* check ranges ... */
if (flags & UVM_FLAG_FIXED) {
UVMHIST_LOG(maphist,"<- VA below map range",0,0,0,0);
@@ -1900,6 +1905,7 @@ uvm_map_findspace(struct vm_map *map, va
hint, vm_map_min(map), vm_map_max(map), 0);
return (NULL);
}
+ orig_hint = hint;
/*
* hint may not be aligned properly; we need round up or down it
CVS commit: [netbsd-9] src/sys/uvm
Module Name:src
Committed By: martin
Date: Sat Apr 1 16:00:29 UTC 2023
Modified Files:
src/sys/uvm [netbsd-9]: uvm_map.c
Log Message:
Pull up following revision(s) (requested by riastradh in ticket #1622):
sys/uvm/uvm_map.c: revision 1.395
uvm(9): Fix 19-year-old bug in assertion about mmap hint.
Previously this would _first_ remember the original hint, and _then_
clamp the hint to the VM map's range:
orig_hint = hint;
if (hint < vm_map_min(map)) { /* check ranges ... */
if (flags & UVM_FLAG_FIXED) {
UVMHIST_LOG(maphist,"<- VA below map range",0,0,0,0);
return (NULL);
}
hint = vm_map_min(map);
...
KASSERTMSG(!topdown || hint <= orig_hint, "hint: %#jx, orig_hint: %#jx",
(uintmax_t)hint, (uintmax_t)orig_hint);
Even if nothing else happens in the ellipsis, taking the branch
guarantees the assertion will fail in the topdown case.
To generate a diff of this commit:
cvs rdiff -u -r1.362.2.2 -r1.362.2.3 src/sys/uvm/uvm_map.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
CVS commit: [netbsd-9] src/sys/uvm
Module Name:src
Committed By: martin
Date: Sat Apr 1 15:54:35 UTC 2023
Modified Files:
src/sys/uvm [netbsd-9]: uvm_mmap.c
Log Message:
Pull up following revision(s) (requested by riastradh in ticket #1621):
sys/uvm/uvm_mmap.c: revision 1.180
mmap(2): If we fail with a hint, try again without it.
`Hint' here means nonzero addr, but no MAP_FIXED or MAP_TRYFIXED.
This is suboptimal -- we could teach uvm_mmap to do a fancier search
using the address as a hint. But this should do for now.
Candidate fix for PR kern/55533.
To generate a diff of this commit:
cvs rdiff -u -r1.172.4.1 -r1.172.4.2 src/sys/uvm/uvm_mmap.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
Modified files:
Index: src/sys/uvm/uvm_mmap.c
diff -u src/sys/uvm/uvm_mmap.c:1.172.4.1 src/sys/uvm/uvm_mmap.c:1.172.4.2
--- src/sys/uvm/uvm_mmap.c:1.172.4.1 Mon Oct 21 20:17:31 2019
+++ src/sys/uvm/uvm_mmap.c Sat Apr 1 15:54:35 2023
@@ -1,4 +1,4 @@
-/* $NetBSD: uvm_mmap.c,v 1.172.4.1 2019/10/21 20:17:31 martin Exp $ */
+/* $NetBSD: uvm_mmap.c,v 1.172.4.2 2023/04/01 15:54:35 martin Exp $ */
/*
* Copyright (c) 1997 Charles D. Cranor and Washington University.
@@ -46,7 +46,7 @@
*/
#include
-__KERNEL_RCSID(0, "$NetBSD: uvm_mmap.c,v 1.172.4.1 2019/10/21 20:17:31 martin Exp $");
+__KERNEL_RCSID(0, "$NetBSD: uvm_mmap.c,v 1.172.4.2 2023/04/01 15:54:35 martin Exp $");
#include "opt_compat_netbsd.h"
#include "opt_pax.h"
@@ -276,7 +276,8 @@ sys_mmap(struct lwp *l, const struct sys
vsize_t size, pageoff, newsize;
vm_prot_t prot, maxprot, extraprot;
int flags, fd, advice;
- vaddr_t defaddr;
+ vaddr_t defaddr = 0; /* XXXGCC */
+ bool addrhint = false;
struct file *fp = NULL;
struct uvm_object *uobj;
int error;
@@ -345,6 +346,12 @@ sys_mmap(struct lwp *l, const struct sys
addr = MAX(addr, defaddr);
else
addr = MIN(addr, defaddr);
+
+ /*
+ * If addr is nonzero and not the default, then the
+ * address is a hint.
+ */
+ addrhint = (addr != 0 && addr != defaddr);
}
/*
@@ -395,11 +402,30 @@ sys_mmap(struct lwp *l, const struct sys
pax_aslr_mmap(l, , orig_addr, flags);
/*
- * now let kernel internal function uvm_mmap do the work.
+ * Now let kernel internal function uvm_mmap do the work.
+ *
+ * If the user provided a hint, take a reference to uobj in
+ * case the first attempt to satisfy the hint fails, so we can
+ * try again with the default address.
*/
-
+ if (addrhint) {
+ if (uobj)
+ (*uobj->pgops->pgo_reference)(uobj);
+ }
error = uvm_mmap(>p_vmspace->vm_map, , size, prot, maxprot,
flags, advice, uobj, pos, p->p_rlimit[RLIMIT_MEMLOCK].rlim_cur);
+ if (addrhint) {
+ if (error) {
+ addr = defaddr;
+ pax_aslr_mmap(l, , orig_addr, flags);
+ error = uvm_mmap(>p_vmspace->vm_map, , size,
+ prot, maxprot, flags, advice, uobj, pos,
+ p->p_rlimit[RLIMIT_MEMLOCK].rlim_cur);
+ } else if (uobj) {
+ /* Release the exta reference we took. */
+ (*uobj->pgops->pgo_detach)(uobj);
+ }
+ }
/* remember to add offset */
*retval = (register_t)(addr + pageoff);
@@ -814,9 +840,12 @@ sys_munlockall(struct lwp *l, const void
* - used by sys_mmap and various framebuffers
* - uobj is a struct uvm_object pointer or NULL for MAP_ANON
* - caller must page-align the file offset
+ *
+ * XXX This appears to leak the uobj in various error branches? Need
+ * to clean up the contract around uobj reference.
*/
-int
+static int
uvm_mmap(struct vm_map *map, vaddr_t *addr, vsize_t size, vm_prot_t prot,
vm_prot_t maxprot, int flags, int advice, struct uvm_object *uobj,
voff_t foff, vsize_t locklimit)
CVS commit: [netbsd-9] src/sys/uvm
Module Name:src Committed By: martin Date: Sat Apr 1 15:54:35 UTC 2023 Modified Files: src/sys/uvm [netbsd-9]: uvm_mmap.c Log Message: Pull up following revision(s) (requested by riastradh in ticket #1621): sys/uvm/uvm_mmap.c: revision 1.180 mmap(2): If we fail with a hint, try again without it. `Hint' here means nonzero addr, but no MAP_FIXED or MAP_TRYFIXED. This is suboptimal -- we could teach uvm_mmap to do a fancier search using the address as a hint. But this should do for now. Candidate fix for PR kern/55533. To generate a diff of this commit: cvs rdiff -u -r1.172.4.1 -r1.172.4.2 src/sys/uvm/uvm_mmap.c Please note that diffs are not public domain; they are subject to the copyright notices on the relevant files.
CVS commit: [netbsd-9] src/sys/uvm
Module Name:src Committed By: martin Date: Mon Nov 11 17:13:28 UTC 2019 Modified Files: src/sys/uvm [netbsd-9]: uvm_fault.c Log Message: Pull up following revision(s) (requested by chs in ticket #414): sys/uvm/uvm_fault.c: revision 1.208 in uvm_fault_lower_io(), fetch all the map entry values that we need before we unlock everything. To generate a diff of this commit: cvs rdiff -u -r1.206 -r1.206.2.1 src/sys/uvm/uvm_fault.c Please note that diffs are not public domain; they are subject to the copyright notices on the relevant files.
CVS commit: [netbsd-9] src/sys/uvm
Module Name:src
Committed By: martin
Date: Mon Nov 11 17:13:28 UTC 2019
Modified Files:
src/sys/uvm [netbsd-9]: uvm_fault.c
Log Message:
Pull up following revision(s) (requested by chs in ticket #414):
sys/uvm/uvm_fault.c: revision 1.208
in uvm_fault_lower_io(), fetch all the map entry values that we need
before we unlock everything.
To generate a diff of this commit:
cvs rdiff -u -r1.206 -r1.206.2.1 src/sys/uvm/uvm_fault.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
Modified files:
Index: src/sys/uvm/uvm_fault.c
diff -u src/sys/uvm/uvm_fault.c:1.206 src/sys/uvm/uvm_fault.c:1.206.2.1
--- src/sys/uvm/uvm_fault.c:1.206 Tue May 28 08:59:35 2019
+++ src/sys/uvm/uvm_fault.c Mon Nov 11 17:13:28 2019
@@ -1,4 +1,4 @@
-/* $NetBSD: uvm_fault.c,v 1.206 2019/05/28 08:59:35 msaitoh Exp $ */
+/* $NetBSD: uvm_fault.c,v 1.206.2.1 2019/11/11 17:13:28 martin Exp $ */
/*
* Copyright (c) 1997 Charles D. Cranor and Washington University.
@@ -32,7 +32,7 @@
*/
#include
-__KERNEL_RCSID(0, "$NetBSD: uvm_fault.c,v 1.206 2019/05/28 08:59:35 msaitoh Exp $");
+__KERNEL_RCSID(0, "$NetBSD: uvm_fault.c,v 1.206.2.1 2019/11/11 17:13:28 martin Exp $");
#include "opt_uvmhist.h"
@@ -1901,11 +1901,18 @@ uvm_fault_lower_io(
int gotpages;
int error;
voff_t uoff;
+ vm_prot_t access_type;
+ int advice;
UVMHIST_FUNC("uvm_fault_lower_io"); UVMHIST_CALLED(maphist);
/* update rusage counters */
curlwp->l_ru.ru_majflt++;
+ /* grab everything we need from the entry before we unlock */
+ uoff = (ufi->orig_rvaddr - ufi->entry->start) + ufi->entry->offset;
+ access_type = flt->access_type & MASK(ufi->entry);
+ advice = ufi->entry->advice;
+
/* Locked: maps(read), amap(if there), uobj */
uvmfault_unlockall(ufi, amap, NULL);
@@ -1915,10 +1922,8 @@ uvm_fault_lower_io(
uvmexp.fltget++;
gotpages = 1;
pg = NULL;
- uoff = (ufi->orig_rvaddr - ufi->entry->start) + ufi->entry->offset;
error = uobj->pgops->pgo_get(uobj, uoff, , ,
- 0, flt->access_type & MASK(ufi->entry), ufi->entry->advice,
- PGO_SYNCIO);
+ 0, access_type, advice, PGO_SYNCIO);
/* locked: pg(if no error) */
/*
CVS commit: [netbsd-9] src/sys/uvm
Module Name:src Committed By: martin Date: Fri Nov 1 18:24:31 UTC 2019 Modified Files: src/sys/uvm [netbsd-9]: uvm_map.c Log Message: Addionally pull up the following revision for ticket #388: sys/uvm/uvm_map.c 1.366 Fix previous; semantics of align argument of uvm_map() is different when UVM_FLAG_COLORMATCH is specified. Should fix PR kern/54669. To generate a diff of this commit: cvs rdiff -u -r1.362.2.1 -r1.362.2.2 src/sys/uvm/uvm_map.c Please note that diffs are not public domain; they are subject to the copyright notices on the relevant files. Modified files: Index: src/sys/uvm/uvm_map.c diff -u src/sys/uvm/uvm_map.c:1.362.2.1 src/sys/uvm/uvm_map.c:1.362.2.2 --- src/sys/uvm/uvm_map.c:1.362.2.1 Fri Nov 1 09:36:32 2019 +++ src/sys/uvm/uvm_map.c Fri Nov 1 18:24:31 2019 @@ -1,4 +1,4 @@ -/* $NetBSD: uvm_map.c,v 1.362.2.1 2019/11/01 09:36:32 martin Exp $ */ +/* $NetBSD: uvm_map.c,v 1.362.2.2 2019/11/01 18:24:31 martin Exp $ */ /* * Copyright (c) 1997 Charles D. Cranor and Washington University. @@ -66,7 +66,7 @@ */ #include -__KERNEL_RCSID(0, "$NetBSD: uvm_map.c,v 1.362.2.1 2019/11/01 09:36:32 martin Exp $"); +__KERNEL_RCSID(0, "$NetBSD: uvm_map.c,v 1.362.2.2 2019/11/01 18:24:31 martin Exp $"); #include "opt_ddb.h" #include "opt_pax.h" @@ -1905,7 +1905,8 @@ uvm_map_findspace(struct vm_map *map, va * hint may not be aligned properly; we need round up or down it * before proceeding further. */ - uvm_map_align_va(, align, topdown); + if ((flags & UVM_FLAG_COLORMATCH) == 0) + uvm_map_align_va(, align, topdown); /* * Look for the first possible address; if there's already
CVS commit: [netbsd-9] src/sys/uvm
Module Name:src Committed By: martin Date: Fri Nov 1 18:24:31 UTC 2019 Modified Files: src/sys/uvm [netbsd-9]: uvm_map.c Log Message: Addionally pull up the following revision for ticket #388: sys/uvm/uvm_map.c 1.366 Fix previous; semantics of align argument of uvm_map() is different when UVM_FLAG_COLORMATCH is specified. Should fix PR kern/54669. To generate a diff of this commit: cvs rdiff -u -r1.362.2.1 -r1.362.2.2 src/sys/uvm/uvm_map.c Please note that diffs are not public domain; they are subject to the copyright notices on the relevant files.
CVS commit: [netbsd-9] src/sys/uvm
Module Name:src Committed By: martin Date: Fri Nov 1 09:36:32 UTC 2019 Modified Files: src/sys/uvm [netbsd-9]: uvm_map.c Log Message: Pull up following revision(s) (requested by rin in ticket #388): sys/uvm/uvm_map.c: revision 1.365 PR kern/54395 - Align hint for virtual address at the beginning of uvm_map() if required. Otherwise, it will be rounded up/down in an unexpected way by uvm_map_space_avail(), which results in assertion failure. Fix kernel panic when executing earm binary (8KB pages) on aarch64 (4KB pages), which relies on mmap(2) with MAP_ALIGNED flag. - Use inline functions/macros consistently. - Add some more KASSERT's. For more details, see the PR as well as discussion on port-kern: http://mail-index.netbsd.org/tech-kern/2019/10/27/msg025629.html To generate a diff of this commit: cvs rdiff -u -r1.362 -r1.362.2.1 src/sys/uvm/uvm_map.c Please note that diffs are not public domain; they are subject to the copyright notices on the relevant files.
CVS commit: [netbsd-9] src/sys/uvm
Module Name:src
Committed By: martin
Date: Fri Nov 1 09:36:32 UTC 2019
Modified Files:
src/sys/uvm [netbsd-9]: uvm_map.c
Log Message:
Pull up following revision(s) (requested by rin in ticket #388):
sys/uvm/uvm_map.c: revision 1.365
PR kern/54395
- Align hint for virtual address at the beginning of uvm_map() if
required. Otherwise, it will be rounded up/down in an unexpected
way by uvm_map_space_avail(), which results in assertion failure.
Fix kernel panic when executing earm binary (8KB pages) on aarch64
(4KB pages), which relies on mmap(2) with MAP_ALIGNED flag.
- Use inline functions/macros consistently.
- Add some more KASSERT's.
For more details, see the PR as well as discussion on port-kern:
http://mail-index.netbsd.org/tech-kern/2019/10/27/msg025629.html
To generate a diff of this commit:
cvs rdiff -u -r1.362 -r1.362.2.1 src/sys/uvm/uvm_map.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
Modified files:
Index: src/sys/uvm/uvm_map.c
diff -u src/sys/uvm/uvm_map.c:1.362 src/sys/uvm/uvm_map.c:1.362.2.1
--- src/sys/uvm/uvm_map.c:1.362 Fri Jul 12 06:27:13 2019
+++ src/sys/uvm/uvm_map.c Fri Nov 1 09:36:32 2019
@@ -1,4 +1,4 @@
-/* $NetBSD: uvm_map.c,v 1.362 2019/07/12 06:27:13 mlelstv Exp $ */
+/* $NetBSD: uvm_map.c,v 1.362.2.1 2019/11/01 09:36:32 martin Exp $ */
/*
* Copyright (c) 1997 Charles D. Cranor and Washington University.
@@ -66,7 +66,7 @@
*/
#include
-__KERNEL_RCSID(0, "$NetBSD: uvm_map.c,v 1.362 2019/07/12 06:27:13 mlelstv Exp $");
+__KERNEL_RCSID(0, "$NetBSD: uvm_map.c,v 1.362.2.1 2019/11/01 09:36:32 martin Exp $");
#include "opt_ddb.h"
#include "opt_pax.h"
@@ -187,6 +187,23 @@ int user_va0_disable = __USER_VA0_DISABL
*/
/*
+ * uvm_map_align_va: round down or up virtual address
+ */
+static __inline void
+uvm_map_align_va(vaddr_t *vap, vsize_t align, int topdown)
+{
+
+ KASSERT(powerof2(align));
+
+ if (align != 0 && (*vap & (align - 1)) != 0) {
+ if (topdown)
+ *vap = rounddown2(*vap, align);
+ else
+ *vap = roundup2(*vap, align);
+ }
+}
+
+/*
* UVM_ET_ISCOMPATIBLE: check some requirements for map entry merging
*/
extern struct vm_map *pager_map;
@@ -1063,6 +1080,7 @@ uvm_map(struct vm_map *map, vaddr_t *sta
int error;
KASSERT((size & PAGE_MASK) == 0);
+ KASSERT((flags & UVM_FLAG_FIXED) == 0 || align == 0);
/*
* for pager_map, allocate the new entry first to avoid sleeping
@@ -1805,13 +1823,9 @@ uvm_map_space_avail(vaddr_t *start, vsiz
*start = ptoa(hint + align); /* adjust to color */
}
}
- } else if (align != 0) {
- if ((*start & (align - 1)) != 0) {
- if (topdown)
-*start &= ~(align - 1);
- else
-*start = roundup(*start, align);
- }
+ } else {
+ KASSERT(powerof2(align));
+ uvm_map_align_va(start, align, topdown);
/*
* XXX Should we PMAP_PREFER() here again?
* eh...i think we're okay
@@ -1861,7 +1875,7 @@ uvm_map_findspace(struct vm_map *map, va
UVMHIST_LOG(maphist, "(map=%#jx, hint=%#jx, len=%ju, flags=%#jx)",
(uintptr_t)map, hint, length, flags);
- KASSERT((flags & UVM_FLAG_COLORMATCH) != 0 || (align & (align - 1)) == 0);
+ KASSERT((flags & UVM_FLAG_COLORMATCH) != 0 || powerof2(align));
KASSERT((flags & UVM_FLAG_COLORMATCH) == 0 || align < uvmexp.ncolors);
KASSERT((flags & UVM_FLAG_FIXED) == 0 || align == 0);
@@ -1888,6 +1902,12 @@ uvm_map_findspace(struct vm_map *map, va
}
/*
+ * hint may not be aligned properly; we need round up or down it
+ * before proceeding further.
+ */
+ uvm_map_align_va(, align, topdown);
+
+ /*
* Look for the first possible address; if there's already
* something at this address, we have to start after it.
*/
CVS commit: [netbsd-9] src/sys/uvm
Module Name:src Committed By: martin Date: Mon Oct 21 20:17:31 UTC 2019 Modified Files: src/sys/uvm [netbsd-9]: uvm_mmap.c Log Message: Pull up following revision(s) (requested by maxv in ticket #355): sys/uvm/uvm_mmap.c: revision 1.173 Change 'npgs' from int to size_t. Otherwise the 64bit->32bit conversion could lead to npgs=0, which is not expected. It later triggers a panic in uvm_vsunlock(). Found by TriforceAFL (Akul Pillai). To generate a diff of this commit: cvs rdiff -u -r1.172 -r1.172.4.1 src/sys/uvm/uvm_mmap.c Please note that diffs are not public domain; they are subject to the copyright notices on the relevant files. Modified files: Index: src/sys/uvm/uvm_mmap.c diff -u src/sys/uvm/uvm_mmap.c:1.172 src/sys/uvm/uvm_mmap.c:1.172.4.1 --- src/sys/uvm/uvm_mmap.c:1.172 Sat Apr 6 03:06:29 2019 +++ src/sys/uvm/uvm_mmap.c Mon Oct 21 20:17:31 2019 @@ -1,4 +1,4 @@ -/* $NetBSD: uvm_mmap.c,v 1.172 2019/04/06 03:06:29 thorpej Exp $ */ +/* $NetBSD: uvm_mmap.c,v 1.172.4.1 2019/10/21 20:17:31 martin Exp $ */ /* * Copyright (c) 1997 Charles D. Cranor and Washington University. @@ -46,7 +46,7 @@ */ #include -__KERNEL_RCSID(0, "$NetBSD: uvm_mmap.c,v 1.172 2019/04/06 03:06:29 thorpej Exp $"); +__KERNEL_RCSID(0, "$NetBSD: uvm_mmap.c,v 1.172.4.1 2019/10/21 20:17:31 martin Exp $"); #include "opt_compat_netbsd.h" #include "opt_pax.h" @@ -132,7 +132,8 @@ sys_mincore(struct lwp *l, const struct vaddr_t start, end, lim; struct vm_map *map; vsize_t len; - int error = 0, npgs; + int error = 0; + size_t npgs; map = >p_vmspace->vm_map;
CVS commit: [netbsd-9] src/sys/uvm
Module Name:src Committed By: martin Date: Mon Oct 21 20:17:31 UTC 2019 Modified Files: src/sys/uvm [netbsd-9]: uvm_mmap.c Log Message: Pull up following revision(s) (requested by maxv in ticket #355): sys/uvm/uvm_mmap.c: revision 1.173 Change 'npgs' from int to size_t. Otherwise the 64bit->32bit conversion could lead to npgs=0, which is not expected. It later triggers a panic in uvm_vsunlock(). Found by TriforceAFL (Akul Pillai). To generate a diff of this commit: cvs rdiff -u -r1.172 -r1.172.4.1 src/sys/uvm/uvm_mmap.c Please note that diffs are not public domain; they are subject to the copyright notices on the relevant files.
