Re: Get rid of UVM_VNODE_CANPERSIST

2022-11-23 Thread Martin Pieuchot
On 23/11/22(Wed) 16:34, Mark Kettenis wrote: > > Date: Wed, 23 Nov 2022 10:52:32 +0100 > > From: Martin Pieuchot > > > > On 22/11/22(Tue) 23:40, Mark Kettenis wrote: > > > > Date: Tue, 22 Nov 2022 17:47:44 + > > > > From: Miod Vallat > >

Re: Get rid of UVM_VNODE_CANPERSIST

2022-11-23 Thread Martin Pieuchot
On 22/11/22(Tue) 23:40, Mark Kettenis wrote: > > Date: Tue, 22 Nov 2022 17:47:44 + > > From: Miod Vallat > > > > > Here is a diff. Maybe bluhm@ can try this on the macppc machine that > > > triggered the original "vref used where vget required" problem? > > > > On a similar machine it

Re: Get rid of UVM_VNODE_CANPERSIST

2022-11-22 Thread Martin Pieuchot
On 18/11/22(Fri) 21:33, Mark Kettenis wrote: > > Date: Thu, 17 Nov 2022 20:23:37 +0100 > > From: Mark Kettenis > > > > > From: Jeremie Courreges-Anglas > > > Date: Thu, 17 Nov 2022 18:00:21 +0100 > > > > > > On Tue, Nov 15 2022, Mar

Get rid of UVM_VNODE_CANPERSIST

2022-11-15 Thread Martin Pieuchot
UVM vnode objects include a reference count to keep track of the number of processes that have the corresponding pages mapped in their VM space. When the last process referencing a given library or executable dies, the reaper will munmap this object on its behalf. When this happens it doesn't

btrace: string comparison in filters

2022-11-11 Thread Martin Pieuchot
Diff below adds support for the common following idiom: syscall:open:entry /comm == "ksh"/ { ... } String comparison is tricky as it can be combined with any other expression in filters, like: syscall:mmap:entry /comm == "cc" && pid != 4589/ { ... } I don't have the energy to

Re: push kernel lock inside ifioctl_get()

2022-11-08 Thread Martin Pieuchot
On 08/11/22(Tue) 15:28, Klemens Nanni wrote: > After this mechanical move, I can unlock the individual SIOCG* in there. I'd suggest grabbing the KERNEL_LOCK() after NET_LOCK_SHARED(). Otherwise you might spin for the first one then release it when going to sleep. > OK? > > Index: if.c >

Mark sched_yield(2) as NOLOCK

2022-11-08 Thread Martin Pieuchot
Now that mmap/munmap/mprotect(2) are no longer creating contention it is possible to see that sched_yield(2) is one of the syscalls waiting for the KERNEL_LOCK() to be released. However this is no longer necessary. Traversing `ps_threads' require either the KERNEL_LOCK() or the SCHED_LOCK() and

Re: xenstore.c: return error number

2022-11-08 Thread Martin Pieuchot
On 01/11/22(Tue) 15:26, Masato Asou wrote: > Hi, > > Return error number instead of call panic(). Makes sense to me. Do you know how this error can occur? Is is a logic error or are we trusting values produced by a third party? > comment, ok? > -- > ASOU Masato > > diff --git

Re: Please test: unlock mprotect/mmap/munmap

2022-11-08 Thread Martin Pieuchot
On 08/11/22(Tue) 11:12, Mark Kettenis wrote: > > Date: Tue, 8 Nov 2022 10:32:14 +0100 > > From: Christian Weisgerber > > > > Martin Pieuchot: > > > > > These 3 syscalls should now be ready to run w/o KERNEL_LOCK(). This > > > will reduce c

Re: push kernel lock down in ifioctl()

2022-11-07 Thread Martin Pieuchot
On 07/11/22(Mon) 15:16, Klemens Nanni wrote: > Not all interface ioctls need the kernel lock, but they all grab it. > > Here's a mechanical diff splitting the single lock/unlock around > ifioctl() into individual lock/unlock dances inside ifioctl(). > > From there we can unlock individual ioctls

Please test: unlock mprotect/mmap/munmap

2022-11-06 Thread Martin Pieuchot
These 3 syscalls should now be ready to run w/o KERNEL_LOCK(). This will reduce contention a lot. I'd be happy to hear from test reports on many architectures and possible workloads. Do not forget to run "make syscalls" before building the kernel. Index: syscalls.master

Re: Towards unlocking mmap(2) & munmap(2)

2022-10-30 Thread Martin Pieuchot
On 30/10/22(Sun) 12:45, Klemens Nanni wrote: > On Sun, Oct 30, 2022 at 12:40:02PM +, Klemens Nanni wrote: > > regress on i386/GENERIC.MP+WITNESS with this diff shows > > Another one; This machine has three read-only NFS mounts, but none of > them are used during builds or regress. It's the

Re: Towards unlocking mmap(2) & munmap(2)

2022-10-30 Thread Martin Pieuchot
On 30/10/22(Sun) 12:40, Klemens Nanni wrote: > On Fri, Oct 28, 2022 at 11:08:55AM +0200, Martin Pieuchot wrote: > > On 20/10/22(Thu) 16:17, Martin Pieuchot wrote: > > > On 11/09/22(Sun) 12:26, Martin Pieuchot wrote: > > > > Diff below adds a minimalist set of ass

Re: Towards unlocking mmap(2) & munmap(2)

2022-10-28 Thread Martin Pieuchot
On 20/10/22(Thu) 16:17, Martin Pieuchot wrote: > On 11/09/22(Sun) 12:26, Martin Pieuchot wrote: > > Diff below adds a minimalist set of assertions to ensure proper locks > > are held in uvm_mapanon() and uvm_unmap_remove() which are the guts of > > mmap(2) for anons and munma

Re: Towards unlocking mmap(2) & munmap(2)

2022-10-20 Thread Martin Pieuchot
On 11/09/22(Sun) 12:26, Martin Pieuchot wrote: > Diff below adds a minimalist set of assertions to ensure proper locks > are held in uvm_mapanon() and uvm_unmap_remove() which are the guts of > mmap(2) for anons and munmap(2). > > Please test it with WITNESS enabled and report bac

Re: Towards unlocking mmap(2) & munmap(2)

2022-09-14 Thread Martin Pieuchot
On 14/09/22(Wed) 15:47, Klemens Nanni wrote: > On 14.09.22 18:55, Mike Larkin wrote: > > On Sun, Sep 11, 2022 at 12:26:31PM +0200, Martin Pieuchot wrote: > > > Diff below adds a minimalist set of assertions to ensure proper locks > > > are held in uvm_mapanon() a

Towards unlocking mmap(2) & munmap(2)

2022-09-11 Thread Martin Pieuchot
Diff below adds a minimalist set of assertions to ensure proper locks are held in uvm_mapanon() and uvm_unmap_remove() which are the guts of mmap(2) for anons and munmap(2). Please test it with WITNESS enabled and report back. Index: uvm/uvm_addr.c

uvm_vnode locking & documentation

2022-09-10 Thread Martin Pieuchot
Previous fix from gnezdo@ pointed out that `u_flags' accesses should be serialized by `vmobjlock'. Diff below documents this and fix the remaining places where the lock isn't yet taken. One exception still remains, the first loop of uvm_vnp_sync(). This cannot be fixed right now due to possible

Re: Unmap page in uvm_anon_release()

2022-09-10 Thread Martin Pieuchot
On 10/09/22(Sat) 15:12, Mark Kettenis wrote: > > Date: Sat, 10 Sep 2022 14:18:02 +0200 > > From: Martin Pieuchot > > > > Diff below fixes a bug exposed when swapping on arm64. When an anon is > > released make sure the all the pmap references to the related

Unmap page in uvm_anon_release()

2022-09-10 Thread Martin Pieuchot
Diff below fixes a bug exposed when swapping on arm64. When an anon is released make sure the all the pmap references to the related page are removed. We could move the pmap_page_protect(pg, PROT_NONE) inside uvm_pagefree() to avoid future issue but that's for a later refactoring. With this

Re: ps(1): add -d (descendancy) option to display parent/child process relationships

2022-09-01 Thread Martin Pieuchot
On 01/09/22(Thu) 03:37, Job Snijders wrote: > Dear all, > > Some ps(1) implementations have an '-d' ('descendancy') option. Through > ASCII art parent/child process relationships are grouped and displayed. > Here is an example: > > $ ps ad -O ppid,user > PID PPID USER TT STAT

Re: pdaemon locking tweak

2022-08-30 Thread Martin Pieuchot
On 30/08/22(Tue) 15:28, Jonathan Gray wrote: > On Mon, Aug 29, 2022 at 01:46:20PM +0200, Martin Pieuchot wrote: > > Diff below refactors the pdaemon's locking by introducing a new *trylock() > > function for a given page. This is shamelessly stolen from NetBSD. > > > >

uvmpd_dropswap()

2022-08-29 Thread Martin Pieuchot
Small refactoring to introduce uvmpd_dropswap(). This will make an upcoming rewrite of the pdaemon smaller & easier to review :o) ok? Index: uvm/uvm_pdaemon.c === RCS file: /cvs/src/sys/uvm/uvm_pdaemon.c,v retrieving revision 1.102

pdaemon locking tweak

2022-08-29 Thread Martin Pieuchot
Diff below refactors the pdaemon's locking by introducing a new *trylock() function for a given page. This is shamelessly stolen from NetBSD. This is part of my ongoing effort to untangle the locks used by the page daemon. ok? Index: uvm//uvm_pdaemon.c

Simplify locking code in pdaemon

2022-08-18 Thread Martin Pieuchot
Use a "slock" variable as done in multiple places to simplify the code. The locking stay the same. This is just a first step to simplify this mess. Also get rid of the return value of the function, it is never checked. ok? Index: uvm/uvm_pdaemon.c

Fix a race in uvm_pseg_release()

2022-08-18 Thread Martin Pieuchot
The lock must be grabbed before iterating on the global array, ok? Index: uvm/uvm_pager.c === RCS file: /cvs/src/sys/uvm/uvm_pager.c,v retrieving revision 1.88 diff -u -p -r1.88 uvm_pager.c --- uvm/uvm_pager.c 15 Aug 2022

Re: uvm_swap: introduce uvm_swap_data_lock

2022-08-17 Thread Martin Pieuchot
On 16/01/22(Sun) 15:35, Martin Pieuchot wrote: > On 30/12/21(Thu) 23:38, Theo Buehler wrote: > > The diff below does two things: it adds a uvm_swap_data_lock mutex and > > trades it for the KERNEL_LOCK in uvm_swapisfull() and uvm_swap_markbad() > > Why is it enough? Whi

Re: patch: change swblk_t type and use it in blist

2022-08-05 Thread Martin Pieuchot
On 05/08/22(Fri) 18:10, Sebastien Marie wrote: > Hi, > > When initially ported blist from DragonFlyBSD, we used custom type bsblk_t > and > bsbmp_t instead of the one used by DragonFlyBSD (swblk_t and u_swblk_t). > > The reason was swblk_t is already defined on OpenBSD, and was incompatible >

Re: Introduce uvm_pagewait()

2022-07-11 Thread Martin Pieuchot
On 28/06/22(Tue) 14:13, Martin Pieuchot wrote: > I'd like to abstract the use of PG_WANTED to start unifying & cleaning > the various cases where a code path is waiting for a busy page. Here's > the first step. > > ok? Anyone? >

Faster M operation for the swapper to be great again

2022-06-30 Thread Martin Pieuchot
Diff below uses two tricks to make uvm_pagermapin/out() faster and less likely to fail in OOM situations. These functions are used to map buffers when swapping pages in/out and when faulting on mmaped files. robert@ even measured a 75% improvement when populating pages related to files that

Re: Use SMR instead of SRP list in rtsock.c

2022-06-30 Thread Martin Pieuchot
On 30/06/22(Thu) 11:56, Claudio Jeker wrote: > On Thu, Jun 30, 2022 at 12:34:33PM +0300, Vitaliy Makkoveev wrote: > > On Thu, Jun 30, 2022 at 11:08:48AM +0200, Claudio Jeker wrote: > > > This diff converts the SRP list to a SMR list in rtsock.c > > > SRP is a bit strange with how it works and the

Re: arp llinfo mutex

2022-06-29 Thread Martin Pieuchot
On 29/06/22(Wed) 19:40, Alexander Bluhm wrote: > Hi, > > To fix the KASSERT(la != NULL) we have to protect the rt_llinfo > with a mutex. The idea is to keep rt_llinfo and RTF_LLINFO consistent. > Also do not put the mutex in the fast path. Losing the RTM_ADD/DELETE race is not a bug. I would

Simplify aiodone daemon

2022-06-29 Thread Martin Pieuchot
The aiodone daemon accounts for and frees/releases pages they were written to swap. It is only used for asynchronous write. The diff below uses this knowledge to: - Stop suggesting that uvm_swap_get() can be asynchronous. There's an assert for PGO_SYNCIO 3 lines above. - Remove unused

Re: Unlocking pledge(2)

2022-06-28 Thread Martin Pieuchot
On 28/06/22(Tue) 18:17, Jeremie Courreges-Anglas wrote: > > Initially I just wandered in syscall_mi.h and found the locking scheme > super weird, even if technically correct. pledge_syscall() better be > safe to call without the kernel lock so I don't understand why we're > sometimes calling it

Re: Fix the swapper

2022-06-28 Thread Martin Pieuchot
On 27/06/22(Mon) 15:44, Martin Pieuchot wrote: > Diff below contain 3 parts that can be committed independently. The 3 > of them are necessary to allow the pagedaemon to make progress in OOM > situation and to satisfy all the allocations waiting for pages in > specific ranges

Introduce uvm_pagewait()

2022-06-28 Thread Martin Pieuchot
I'd like to abstract the use of PG_WANTED to start unifying & cleaning the various cases where a code path is waiting for a busy page. Here's the first step. ok? Index: uvm/uvm_amap.c === RCS file: /cvs/src/sys/uvm/uvm_amap.c,v

Re: kernel lock in arp

2022-06-27 Thread Martin Pieuchot
On 27/06/22(Mon) 19:11, Alexander Bluhm wrote: > On Mon, Jun 27, 2022 at 11:49:23AM +0200, Alexander Bluhm wrote: > > On Sat, May 21, 2022 at 10:50:28PM +0300, Vitaliy Makkoveev wrote: > > > This diff looks good, except the re-check after kernel lock. It???s > > > supposed `rt??? could became

CoW & neighbor pages

2022-06-27 Thread Martin Pieuchot
When faulting a page after COW neighborhood pages are likely to already be entered. So speed up the fault by doing a narrow fault (do not try to map in adjacent pages). This is stolen from NetBSD. ok? Index: uvm/uvm_fault.c ===

Fix the swapper

2022-06-27 Thread Martin Pieuchot
Diff below contain 3 parts that can be committed independently. The 3 of them are necessary to allow the pagedaemon to make progress in OOM situation and to satisfy all the allocations waiting for pages in specific ranges. * uvm/uvm_pager.c part reserves a second segment for the page daemon.

pdaemon: reserve memory for swapping

2022-06-26 Thread Martin Pieuchot
uvm_swap_io() needs to perform up to 4 allocations to write pages to disk. In OOM situation uvm_swap_allocpages() always fail because the kernel doesn't reserve enough pages. Diff below set `uvmexp.reserve_pagedaemon' to the number of pages needed to write a cluster of pages to disk. With this

Re: set RTF_DONE in sysctl_dumpentry for the routing table

2022-06-08 Thread Martin Pieuchot
On 08/06/22(Wed) 16:13, Claudio Jeker wrote: > Notice while hacking in OpenBGPD. Unlike routing socket messages the > messages from the sysctl interface have RTF_DONE not set. > I think it would make sense to set RTF_DONE also in this case since it > makes reusing code easier. > > All messages

Re: Fix clearing of sleep timeouts

2022-06-06 Thread Martin Pieuchot
On 06/06/22(Mon) 06:47, David Gwynne wrote: > On Sun, Jun 05, 2022 at 03:57:39PM +, Visa Hankala wrote: > > On Sun, Jun 05, 2022 at 12:27:32PM +0200, Martin Pieuchot wrote: > > > On 05/06/22(Sun) 05:20, Visa Hankala wrote: > > > > Encountered the following panic:

Re: Fix clearing of sleep timeouts

2022-06-05 Thread Martin Pieuchot
On 05/06/22(Sun) 05:20, Visa Hankala wrote: > Encountered the following panic: > > panic: kernel diagnostic assertion "(p->p_flag & P_TIMEOUT) == 0" failed: > file "/usr/src/sys/kern/kern_synch.c", line 373 > Stopped at db_enter+0x10: popq%rbp > TIDPIDUID PRFLAGS

Re: start unlocking kbind(2)

2022-05-31 Thread Martin Pieuchot
On 18/05/22(Wed) 15:53, Alexander Bluhm wrote: > On Tue, May 17, 2022 at 10:44:54AM +1000, David Gwynne wrote: > > + cookie = SCARG(uap, proc_cookie); > > + if (pr->ps_kbind_addr == pc) { > > + membar_datadep_consumer(); > > + if (pr->ps_kbind_cookie != cookie) > > +

Re: ffs_truncate: Missing uvm_vnp_uncache() w/ softdep

2022-05-24 Thread Martin Pieuchot
On 24/05/22(Tue) 15:24, Mark Kettenis wrote: > > Date: Tue, 24 May 2022 14:28:39 +0200 > > From: Martin Pieuchot > > > > The softdep code path is missing a UVM cache invalidation compared to > > the !softdep one. This is necessary to flush pages of a persis

Please test: rewrite of pdaemon

2022-05-24 Thread Martin Pieuchot
Diff below brings in & adapt most of the changes from NetBSD's r1.37 of uvm_pdaemon.c. My motivation for doing this is to untangle the inner loop of uvmpd_scan_inactive() which will allow us to split the global `pageqlock' mutex in a next step. The idea behind this change is to get rid of the

ffs_truncate: Missing uvm_vnp_uncache() w/ softdep

2022-05-24 Thread Martin Pieuchot
The softdep code path is missing a UVM cache invalidation compared to the !softdep one. This is necessary to flush pages of a persisting vnode. Since uvm_vnp_setsize() is also called later in this function for the !softdep case move it to not call it twice. ok? Index: ufs/ffs/ffs_inode.c

Re: Call uvm_vnp_uncache() before VOP_RENAME()

2022-05-24 Thread Martin Pieuchot
On 17/05/22(Tue) 16:55, Martin Pieuchot wrote: > nfsrv_rename() should behave like dorenameat() and tell UVM to "flush" a > possibly mmap'ed file before calling VOP_RENAME(). > > ok? Anyone? > Index: nfs/nfs_serv.c > ==

Re: start unlocking kbind(2)

2022-05-17 Thread Martin Pieuchot
On 17/05/22(Tue) 10:44, David Gwynne wrote: > this narrows the scope of the KERNEL_LOCK in kbind(2) so the syscall > argument checks can be done without the kernel lock. > > care is taken to allow the pc/cookie checks to run without any lock by > being careful with the order of the checks. all

Call uvm_vnp_uncache() before VOP_RENAME()

2022-05-17 Thread Martin Pieuchot
nfsrv_rename() should behave like dorenameat() and tell UVM to "flush" a possibly mmap'ed file before calling VOP_RENAME(). ok? Index: nfs/nfs_serv.c === RCS file: /cvs/src/sys/nfs/nfs_serv.c,v retrieving revision 1.120 diff -u -p

Re: uvm_pagedequeue()

2022-05-12 Thread Martin Pieuchot
On 10/05/22(Tue) 20:23, Mark Kettenis wrote: > > Date: Tue, 10 May 2022 18:45:21 +0200 > > From: Martin Pieuchot > > > > On 05/05/22(Thu) 14:54, Martin Pieuchot wrote: > > > Diff below introduces a new wrapper to manipulate active/inactive page > > >

Re: uvm_pagedequeue()

2022-05-10 Thread Martin Pieuchot
On 05/05/22(Thu) 14:54, Martin Pieuchot wrote: > Diff below introduces a new wrapper to manipulate active/inactive page > queues. > > ok? Anyone? > Index: uvm/uvm_page.c > === > RCS file: /cvs/src/sys/uvm/uvm_

Re: uvm: Consider BUFPAGES_DEFICIT in swap_shortage

2022-05-09 Thread Martin Pieuchot
On 05/05/22(Thu) 10:56, Bob Beck wrote: > On Thu, May 05, 2022 at 10:16:23AM -0600, Bob Beck wrote: > > Ugh. You???re digging in the most perilous parts of the pile. > > > > I will go look with you??? sigh. (This is not yet an ok for that.) > > > > > On May

uvm: Consider BUFPAGES_DEFICIT in swap_shortage

2022-05-05 Thread Martin Pieuchot
When considering the amount of free pages in the page daemon a small amount is always kept for the buffer cache... except in one place. The diff below gets rid of this exception. This is important because uvmpd_scan() is called conditionally using the following check: if (uvmexp.free -

uvm_pagedequeue()

2022-05-05 Thread Martin Pieuchot
Diff below introduces a new wrapper to manipulate active/inactive page queues. ok? Index: uvm/uvm_page.c === RCS file: /cvs/src/sys/uvm/uvm_page.c,v retrieving revision 1.165 diff -u -p -r1.165 uvm_page.c --- uvm/uvm_page.c 4

Merge swap-backed and object-backed inactive lists

2022-05-02 Thread Martin Pieuchot
Let's simplify the existing logic and use a single list for inactive pages. uvmpd_scan_inactive() already does a lot of check if it finds a page which is swap-backed. This will be improved in a next change. ok? Index: uvm/uvm.h

uvmpd_scan(): Recheck PG_BUSY after locking the page

2022-04-28 Thread Martin Pieuchot
rw_enter(9) can sleep. When the lock is finally acquired by the pagedaemon the previous check might no longer be true and the page could be busy. In this case we shouldn't touch it. Diff below recheck for PG_BUSY after acquiring the lock and also use a variable for the lock to reduce the

Call uvm_pageactivate() from uvm_pageunwire()

2022-04-26 Thread Martin Pieuchot
I'd like to use a proper interface to add/remove pages on the active/inactive queues. This will help for lock assertions and help improving the existing LRU limitations. Diff below makes uvm_pageunwire() call uvm_pageactivate() instead of inserting the page itself. ok? Index: uvm/uvm_page.c

Decrement uvmexp.swpgonly

2022-04-26 Thread Martin Pieuchot
Small diff to decrement the counter only if the I/O succeed. This prevent a false positive if a check is performed before an error is returned. ok? Index: uvm/uvm_swap.c === RCS file: /cvs/src/sys/uvm/uvm_swap.c,v retrieving

Re: refcount btrace

2022-04-11 Thread Martin Pieuchot
On 08/04/22(Fri) 12:16, Alexander Bluhm wrote: > On Fri, Apr 08, 2022 at 02:39:34AM +, Visa Hankala wrote: > > On Thu, Apr 07, 2022 at 07:55:11PM +0200, Alexander Bluhm wrote: > > > On Wed, Mar 23, 2022 at 06:13:27PM +0100, Alexander Bluhm wrote: > > > > In my opinion tracepoints give insight

Kill selrecord()

2022-04-11 Thread Martin Pieuchot
Now that poll(2) & select(2) use the kqueue backend under the hood we can start retiring the old machinery. The diff below does not touch driver definitions, however it : - kills selrecord() & doselwakeup() - make it obvious that `kern.nselcoll' is now always 0 - Change all poll/select hooks

Re: refcount btrace

2022-03-21 Thread Martin Pieuchot
On 20/03/22(Sun) 05:39, Visa Hankala wrote: > On Sat, Mar 19, 2022 at 12:10:11AM +0100, Alexander Bluhm wrote: > > On Thu, Mar 17, 2022 at 07:25:27AM +, Visa Hankala wrote: > > > On Thu, Mar 17, 2022 at 12:42:13AM +0100, Alexander Bluhm wrote: > > > > I would like to use btrace to debug

Swap encrypt under memory pressure

2022-03-12 Thread Martin Pieuchot
Try to allocate the buffer before doing the encryption, if it fails we do not spend time doing the encryption. This reduce the pressure when swapping with low memory. ok? Index: uvm/uvm_swap.c === RCS file:

Re: more MAKEDEV cleanup

2022-02-10 Thread Martin Pieuchot
On 05/04/21(Mon) 09:25, Miod Vallat wrote: > The following diff attempts to clean up a few loose ends in the current > MAKEDEV files: > > - remove no-longer applicable device definitions (MSCP and SMD disks, > this kind of thing). > - makes sure all platforms use the same `ramdisk' target for >

Re: uvm_unmap_kill_entry(): unwire with map lock held

2022-02-04 Thread Martin Pieuchot
On 04/02/22(Fri) 03:39, Klemens Nanni wrote: > [...] > ... with the lock grabbed in uvm_map_teardown() that is, otherwise > the first call path can lock against itself (regress/misc/posixtestsuite > is a reproduce for this): > > vm_map_lock_read_ln+0x38 > uvm_fault_unwire+0x58 >

Re: uvm_unmap_kill_entry(): unwire with map lock held

2022-01-31 Thread Martin Pieuchot
On 31/01/22(Mon) 10:24, Klemens Nanni wrote: > Running with my uvm assert diff showed that uvm_fault_unwire_locked() > was called without any locks held. > > This happened when I rebooted my machine: > > uvm_fault_unwire_locked() > uvm_unmap_kill_entry_withlock() >

Re: unlock mmap(2) for anonymous mappings

2022-01-24 Thread Martin Pieuchot
On 24/01/22(Mon) 12:06, Klemens Nanni wrote: > On Sun, Jan 16, 2022 at 09:22:50AM -0300, Martin Pieuchot wrote: > > IMHO this approach of let's try if it works now and revert if it isn't > > doesn't help us make progress. I'd be more confident seeing diffs that > > asse

Re: DDBPROF: move option to amd64,i386 GENERIC

2022-01-18 Thread Martin Pieuchot
On 18/01/22(Tue) 04:38, Klemens Nanni wrote: > While intended for more architectures, DDBPROF is strictly amd64 and > i386 only, so the machine-independent sys/conf/GENERIC does not seem fit > (until all architectures are supported). This define should die. There's no need to polish this turd.

Re: uvm_swap: introduce uvm_swap_data_lock

2022-01-16 Thread Martin Pieuchot
Nice! On 30/12/21(Thu) 23:38, Theo Buehler wrote: > The diff below does two things: it adds a uvm_swap_data_lock mutex and > trades it for the KERNEL_LOCK in uvm_swapisfull() and uvm_swap_markbad() Why is it enough? Which fields is the lock protecting in these function? Is it `uvmexp.swpages',

Re: unlock mmap(2) for anonymous mappings

2022-01-16 Thread Martin Pieuchot
On 14/01/22(Fri) 23:01, Mark Kettenis wrote: > > Date: Tue, 11 Jan 2022 23:13:20 + > > From: Klemens Nanni > > > > On Tue, Jan 11, 2022 at 09:54:44AM -0700, Theo de Raadt wrote: > > > > Now this is clearly a "slow" path. I don't think there is any reason > > > > not to put all the code in

Re: patch: add a new ktrace facility for replacing some printf-debug

2022-01-07 Thread Martin Pieuchot
On 07/01/22(Fri) 10:54, Sebastien Marie wrote: > Hi, > > Debugging some code paths is complex: for example, unveil(2) code is > involved inside VFS, and using DEBUG_UNVEIL means that the kernel is > spamming printf() for all processes using unveil(2) (a lot of > processes) instead of just the

Re: gprof: Profiling a multi-threaded application

2021-12-10 Thread Martin Pieuchot
On 10/12/21(Fri) 09:56, Yuichiro NAITO wrote: > Any comments about this topic? I'm ok with this approach. I would appreciate if somebody else could take it over, I'm too busy with other stuff. > On 7/12/21 18:09, Yuichiro NAITO wrote: > > Hi, Martin > > > > n 2021/07/1

Re: net write in pcb hash

2021-12-09 Thread Martin Pieuchot
On 08/12/21(Wed) 22:39, Alexander Bluhm wrote: > On Wed, Dec 08, 2021 at 03:28:34PM -0300, Martin Pieuchot wrote: > > On 04/12/21(Sat) 01:02, Alexander Bluhm wrote: > > > Hi, > > > > > > As I want a read-only net lock for forwarding, all paths should be &g

Re: net write in pcb hash

2021-12-08 Thread Martin Pieuchot
On 04/12/21(Sat) 01:02, Alexander Bluhm wrote: > Hi, > > As I want a read-only net lock for forwarding, all paths should be > checked for the correct net lock and asserts. I found code in > in_pcbhashlookup() where reading the PCB table results in a write > to optimize the cache. > > Porperly

Re: kbind(2): push kernel lock down as needed

2021-12-08 Thread Martin Pieuchot
On 06/12/21(Mon) 14:58, Scott Cheloha wrote: > On Mon, Dec 06, 2021 at 08:35:15PM +0100, Mark Kettenis wrote: > > > Date: Sun, 5 Dec 2021 18:01:04 -0600 > > > From: Scott Cheloha > > > > > > Two things in sys_kbind() need an explicit kernel lock. First, > > > sigexit(). Second,

Re: Rework UNIX sockets locking to be fine grained

2021-12-01 Thread Martin Pieuchot
On 24/11/21(Wed) 15:04, Vitaliy Makkoveev wrote: > [...] > Really, this is the simplest way. The shared lock for the pair of > sockets moves re-lock dances to the connect and disconnect stages which > should be also protected by locks. And not for the pair. Many SOCK_DGRAM > sockets could be

Re: Please test: UVM fault unlocking (aka vmobjlock)

2021-11-29 Thread Martin Pieuchot
On 24/11/21(Wed) 11:16, Martin Pieuchot wrote: > Diff below unlock the bottom part of the UVM fault handler. I'm > interested in squashing the remaining bugs. Please test with your usual > setup & report back. Thanks to all the testers, here's a new version that includes a bu

Re: Rework UNIX sockets locking to be fine grained

2021-11-24 Thread Martin Pieuchot
On 22/11/21(Mon) 14:42, Vitaliy Makkoveev wrote: > On Sat, Nov 20, 2021 at 03:12:31AM +0300, Vitaliy Makkoveev wrote: > > Updated diff. Re-lock dances were simplified in the unix(4) sockets > > layer. > > > > Reference counters added to unix(4) sockets layer too. This makes > > pointer

Please test: UVM fault unlocking (aka vmobjlock)

2021-11-24 Thread Martin Pieuchot
Diff below unlock the bottom part of the UVM fault handler. I'm interested in squashing the remaining bugs. Please test with your usual setup & report back. Thanks, Martin diff --git sys/arch/amd64/conf/GENERIC.MP sys/arch/amd64/conf/GENERIC.MP index bb842f6d96e..e5334c19eac 100644 ---

Re: Retry sleep in poll/select

2021-11-18 Thread Martin Pieuchot
On 17/11/21(Wed) 09:51, Scott Cheloha wrote: > > On Nov 17, 2021, at 03:22, Martin Pieuchot wrote: > > > > ´╗┐On 16/11/21(Tue) 13:55, Visa Hankala wrote: > >> Currently, dopselect() and doppoll() call tsleep_nsec() without retry. > >> cheloha@ asked if the func

Re: Retry sleep in poll/select

2021-11-17 Thread Martin Pieuchot
On 16/11/21(Tue) 13:55, Visa Hankala wrote: > Currently, dopselect() and doppoll() call tsleep_nsec() without retry. > cheloha@ asked if the functions should handle spurious wakeups. I guess > such wakeups are unlikely with the nowake wait channel, but I am not > sure if that is a safe guess. I'm

Re: bt.5 document count()

2021-11-16 Thread Martin Pieuchot
On 16/11/21(Tue) 11:07, Claudio Jeker wrote: > This documents count(). This function only works when used like this > @map[key] = count(); > But it is implemented and works. If used differently you get a syntax > error which is not helpful. This is why I chose to document it like this. >

Re: poll/select: Lazy removal of knotes

2021-11-06 Thread Martin Pieuchot
On 06/11/21(Sat) 15:53, Visa Hankala wrote: > On Fri, Nov 05, 2021 at 10:04:50AM +0100, Martin Pieuchot wrote: > > New poll/select(2) implementation convert 'struct pollfd' and 'fdset' to > > knotes (kqueue event descriptors) then pass them to the kqueue subsystem. > >

poll/select: Lazy removal of knotes

2021-11-05 Thread Martin Pieuchot
New poll/select(2) implementation convert 'struct pollfd' and 'fdset' to knotes (kqueue event descriptors) then pass them to the kqueue subsystem. A knote is allocated, with kqueue_register(), for every read, write and except condition watched on a given FD. That means at most 3 allocations might

Re: UNIX sockets: use vnode(9) lock to protect `v_socket' dereference

2021-11-05 Thread Martin Pieuchot
On 26/10/21(Tue) 14:12, Vitaliy Makkoveev wrote: > Another step to make UNIX sockets locking fine grained. > > The listening socket has the references from file descriptors layer and > from the vnode(9) layer. This means when we close(2)'ing such socket it > still referenced by concurrent thread

Re: UNIX sockets: make `unp_rights', `unp_msgcount' and `unp_file' atomic

2021-11-05 Thread Martin Pieuchot
On 30/10/21(Sat) 21:22, Vitaliy Makkoveev wrote: > This completely removes global rwlock(9) from the unp_internalize() and > unp_externalize() normal paths but only leaves it in unp_externalize() > error path. Also we don't need to simultaneously hold both fdplock() > and `unp_lock' in

Re: Please test: full poll/select(2) switch

2021-10-29 Thread Martin Pieuchot
On 29/10/21(Fri) 14:48, Alexandre Ratchov wrote: > On Fri, Oct 29, 2021 at 01:12:06PM +0100, Martin Pieuchot wrote: > > On 29/10/21(Fri) 13:12, Alexandre Ratchov wrote: > > > On Sat, Oct 23, 2021 at 10:40:56AM +0100, Martin Pieuchot wrote: > > > > Diff below swit

Re: Please test: full poll/select(2) switch

2021-10-29 Thread Martin Pieuchot
On 29/10/21(Fri) 13:12, Alexandre Ratchov wrote: > On Sat, Oct 23, 2021 at 10:40:56AM +0100, Martin Pieuchot wrote: > > Diff below switches both poll(2) and select(2) to the kqueue-based > > implementation. > > > > In addition it switches libevent(3) to use poll(2)

Re: uvm_km_pgremove() tweak

2021-10-24 Thread Martin Pieuchot
On 24/10/21(Sun) 14:49, Martin Pieuchot wrote: > Here's another small tweak I could extract from the UVM unlocking diff. > This doesn't introduce any functional change. uvm_km_pgremove() is used > in only one place. Updated diff that also moves pmap_kremove() into the intrsaf

uvm_km_pgremove() tweak

2021-10-24 Thread Martin Pieuchot
Here's another small tweak I could extract from the UVM unlocking diff. This doesn't introduce any functional change. uvm_km_pgremove() is used in only one place. Ok? Index: uvm/uvm_km.c === RCS file: /cvs/src/sys/uvm/uvm_km.c,v

More uvm_obj_destroy()

2021-10-23 Thread Martin Pieuchot
Diff below is extracted from the current UVM unlocking diff. It adds a couple of uvm_obj_destroy() and move some uvm_obj_init() around. uvm_obj_destroy() will be used to release the memory of the, possibly shared, lock allocated in uvm_obj_init(). When it is call the object should no longer

Please test: full poll/select(2) switch

2021-10-23 Thread Martin Pieuchot
Diff below switches both poll(2) and select(2) to the kqueue-based implementation. In addition it switches libevent(3) to use poll(2) by default for testing purposes. I don't have any open bug left with this diff and I'm happily running GNOME with it. So I'd be happy if you could try to break

Re: xhci uhub on arm64: handle device in SS_INACTIVE state

2021-10-22 Thread Martin Pieuchot
On 17/10/21(Sun) 09:06, Christopher Zimmermann wrote: > Hi, > > on my RK3399, a usb device connected to the USB 3 port is not detected > during boot because it is in SS_INACTIVE (0x00c0) state: > > uhub3 at usb3 configuration 1 interface 0 "Generic xHCI root hub" rev > 3.00/1.00 addr 1 > uhub3:

Re: Make pipe event filters MP-safe

2021-10-22 Thread Martin Pieuchot
On 22/10/21(Fri) 13:15, Visa Hankala wrote: > This diff makes pipe event filters ready to run without the kernel lock. > The code pattern in the callbacks is the same as in sockets. Pipes > have a klist lock already. > > So far, pipe event filters have used read-locking. The patch changes > that

Re: Set klist lock for sockets, v2

2021-10-22 Thread Martin Pieuchot
On 22/10/21(Fri) 13:11, Visa Hankala wrote: > Here is another attempt to set klist lock for sockets. This is a revised > version of a patch that I posted in January [1]. > > Using solock() for the klists is probably the easiest way at the time > being. However, the lock is a potential point of

POLLHUP vs EVFILT_EXCEPT semantic

2021-10-22 Thread Martin Pieuchot
Last year we added the new EVFILT_EXCEPT filter type to kqueue in order to report conditions currently available via POLLPRI/POLLRDBAND in poll(2) and select(2). This new filter has been implemented in tty and socket by re-using the existing kqueue's "read" filter. This has a downside which is

Re: vnode lock: remove VLOCKSWORK flag

2021-10-15 Thread Martin Pieuchot
On 15/10/21(Fri) 09:27, Sebastien Marie wrote: > Hi, > > The following diff removes VLOCKSWORK flag. Nice. > This flag is currently used to mark or unmark a vnode to actively > check vnode locking semantic (when compiled with VFSLCKDEBUG). > > Currently, VLOCKSWORK flag isn't properly set for

poll(2) on top of kqueue

2021-10-14 Thread Martin Pieuchot
Diff below is the counterpart of the select(2) one I just committed to make poll(2) and ppoll(2) use kqueue internally. They use the same logic as select(2): convert pollfd into kqueue events with EV_SET(2) then wait in kqueue_scan(). To make this implementation compatible with the existing

Re: Switch to kqueue based select(2)

2021-10-13 Thread Martin Pieuchot
On 13/10/21(Wed) 11:41, Alexander Bluhm wrote: > On Sat, Oct 02, 2021 at 09:10:13AM +0200, Martin Pieuchot wrote: > > ok? > > OK bluhm@ > > > + /* Maxium number of events per iteration */ > > Maximum > > > +int > > +pselcollect(struct p

Re: mi_switch() & setting `p_stat'

2021-10-03 Thread Martin Pieuchot
On 02/10/21(Sat) 21:09, Mark Kettenis wrote: > > Date: Sat, 2 Oct 2021 20:35:41 +0200 > > From: Martin Pieuchot > > [...] > > There's no sleeping point but a call to wakeup(). This wakeup() is > > supposed to wake a btrace(8) process. But if the curpro

  1   2   3   4   5   6   7   8   9   10   >