Re: uvm: __inline -> inline

2020-09-22 Thread Martin Pieuchot
On 22/09/20(Tue) 10:20, Mark Kettenis wrote: > > Date: Tue, 22 Sep 2020 09:15:00 +0200 > > From: Martin Pieuchot > > > > Spell inline correctly, also reduce the diff with NetBSD for uvm_amap.c > > and uvm_fault.c. > > > > ok? > > In g

pmap_enter(9) doesn't sleep

2020-09-22 Thread Martin Pieuchot
Allocations in the various pmap_enter(9) are done with uvm_pagealloc(9), which sets the UVM_PLA_NOWAIT flag, and/or with pool_get(9) w/ PR_NOWAIT. So the comment below seems outdated to me, ok to kill it? Index: uvm/uvm_fault.c ===

uvm: __inline -> inline

2020-09-22 Thread Martin Pieuchot
Spell inline correctly, also reduce the diff with NetBSD for uvm_amap.c and uvm_fault.c. ok? Index: uvm/uvm_addr.c === RCS file: /cvs/src/sys/uvm/uvm_addr.c,v retrieving revision 1.28 diff -u -p -r1.28 uvm_addr.c --- uvm/uvm_addr.c

Re: sigabort(), p_sigmask & p_siglist

2020-09-16 Thread Martin Pieuchot
On 16/09/20(Wed) 02:08, Theo de Raadt wrote: > Something doesn't feel right. > > db_kill_cmd finds a process, called p, then kills it. In your new diff > calling sigexit, take note of the comment at the top: > > * Force the current process to exit with the specified signal, dumping core > >

Re: KASSERT() for VOP_*

2020-09-16 Thread Martin Pieuchot
On 09/09/20(Wed) 08:41, Martin Pieuchot wrote: > This is mostly the same diff that has been backed out months ago with > the VOP_CLOSE() case fixed. VOP_CLOSE() can accept a NULL argument > instead of `curproc' when garbage collecting passed FDs. > > The intent is to stop passing

Re: sigabort(), p_sigmask & p_siglist

2020-09-16 Thread Martin Pieuchot
On 16/09/20(Wed) 06:09, Miod Vallat wrote: > > > Diff below introduces an helper for sending an uncatchable SIGABRT and > > annotate that `p_siglist' and `p_sigmask' are updated using atomic > > operations. > > Why not use sigexit(p, SIGABRT); for that purpose? That's a better solution indeed.

sigabort(), p_sigmask & p_siglist

2020-09-15 Thread Martin Pieuchot
Diff below introduces an helper for sending an uncatchable SIGABRT and annotate that `p_siglist' and `p_sigmask' are updated using atomic operations. As a result setsigvec() becomes local to kern/kern_sig.c. Note that other places in the kernel use sigexit(p, SIGABRT) for the same purpose and

curproc vs MP vs locking

2020-09-15 Thread Martin Pieuchot
Many functions in the kernel take a "struct proc *" as argument. When reviewing diffs or reading the signature of such functions it is not clear if this pointer can be any thread or if it is, like in many cases, pointing to `curproc'. This distinction matters when it comes to reading/writing

Re: go/rust vs uvm_map_inentry()

2020-09-13 Thread Martin Pieuchot
On 13/09/20(Sun) 16:54, Mark Kettenis wrote: > > Date: Sun, 13 Sep 2020 16:49:48 +0200 > > From: Sebastien Marie > > > > On Sun, Sep 13, 2020 at 03:29:57PM +0200, Martin Pieuchot wrote: > > > I'm no longer able to reproduce the corruption while building

Re: pppoe: move softc list out of NET_LOCK() into new pppoe lock

2020-09-13 Thread Martin Pieuchot
On 13/09/20(Sun) 15:12, Klemens Nanni wrote: > This is my first try trading global locks for interface specific ones. > > pppoe(4) keeps a list of all its interfaces which is then obviously > traversed during create and destroy. > > Currently, the net lock is grabbed for this, but there seems to

go/rust vs uvm_map_inentry()

2020-09-13 Thread Martin Pieuchot
I'm no longer able to reproduce the corruption while building lang/go with the diff below. Something relevant to threading change in go since march? Can someone try this diff and tell me if go and/or rust still fail? Index: uvm/uvm_map.c

Re: pppoe: start documenting locks

2020-09-13 Thread Martin Pieuchot
On 13/09/20(Sun) 10:05, Klemens Nanni wrote: > > Here's a start at struct pppoe_softc; for every member I went through > code paths looking for *_LOCK() or *_ASSERT_LOCKED(). > > Pretty much all members are under the net lock, some are proctected by > both net and kernel lock, e.g. the start

Re: sppp: add free() sizes

2020-09-12 Thread Martin Pieuchot
On 12/09/20(Sat) 14:49, Klemens Nanni wrote: > These are the last free(buf, 0) occurences in if_pppoe.c and > if_spppsubr.c changing to non-zero sizes. > > I've been running with this the last week without any issues. > > Feedback? OK? Maybe store `pwdlen' and `idlen' in "struct sppp" instead

UVM tracepoints for dt(4)

2020-09-11 Thread Martin Pieuchot
To investigate the race exposed by the last locking change in uvm_map_inentry() [0], I'd like to add the following tracepoints. The idea is to compare page fault addresses and permissions with the insertion/removal of entries in a given map. Diff below is the first part of the puzzle, ok? [0]

Re: issignal() w/o KERNEL_LOCK()

2020-09-09 Thread Martin Pieuchot
On 09/09/20(Wed) 10:02, Claudio Jeker wrote: > On Wed, Sep 09, 2020 at 08:33:30AM +0200, Martin Pieuchot wrote: > > Per-process data structures needed to suspend the execution of threads > > are since recently protected by the SCHED_LOCK(). So the KERNEL_LOCK() > >

KASSERT() for VOP_*

2020-09-09 Thread Martin Pieuchot
This is mostly the same diff that has been backed out months ago with the VOP_CLOSE() case fixed. VOP_CLOSE() can accept a NULL argument instead of `curproc' when garbage collecting passed FDs. The intent is to stop passing a "struct proc *" when a function applies only to `curproc'.

sigismasked()

2020-09-09 Thread Martin Pieuchot
Simple helper function to centralize the manipulation of `ps_sigignore' and `p_sigmask' in kern/kern_sig.c and later on add the corresponding asserts, ok? Index: kern/kern_sig.c === RCS file: /cvs/src/sys/kern/kern_sig.c,v retrieving

issignal() w/o KERNEL_LOCK()

2020-09-09 Thread Martin Pieuchot
Per-process data structures needed to suspend the execution of threads are since recently protected by the SCHED_LOCK(). So the KERNEL_LOCK() dance inside issignal() is no longer necessary and can be removed, ok? Note that CURSIG() is currently always called with the KERNEL_LOCK() held so the

m_defrag(9) leak

2020-08-25 Thread Martin Pieuchot
Maxime Villard mentioned a leak due to a missing m_freem() in wg(4): https://marc.info/?l=netbsd-tech-net=159827988018641=2 It seems to be that such leak is present in other uses of m_defrag() in the tree. I won't take the time to go though all of them, but an audit would be welcome.

Enable EVFILT_EXCEPT

2020-08-21 Thread Martin Pieuchot
The kqueue-based poll(2) backend is still a WIP due to regressions in the kqueue layer. In the meantime should we expose EVFILT_EXCEPT to userland? The diff below should be enough to allow userland apps to use the new code paths. ok? Index: sys/event.h

Re: Fewer pool_get() in kqueue_register()

2020-08-19 Thread Martin Pieuchot
On 18/08/20(Tue) 15:30, Visa Hankala wrote: > On Tue, Aug 18, 2020 at 11:04:47AM +0200, Martin Pieuchot wrote: > > Diff below changes the order of operations in kqueue_register() to get > > rid of an unnecessary pool_get(). When an event is already present on > > the list tr

Re: Fewer pool_get() in kqueue_register()

2020-08-18 Thread Martin Pieuchot
On 18/08/20(Tue) 11:22, Mark Kettenis wrote: > > Date: Tue, 18 Aug 2020 11:04:47 +0200 > > From: Martin Pieuchot > > > > Diff below changes the order of operations in kqueue_register() to get > > rid of an unnecessary pool_get(). When an event is already present

Push KERNEL_LOCK/UNLOCK in trapsignal()

2020-08-18 Thread Martin Pieuchot
Taken from a larger diff from claudio@, this reduces the lock dances in MD code and put it where we should focus our effort in kern/kern_sig.c. ok? Index: kern/kern_sig.c === RCS file: /cvs/src/sys/kern/kern_sig.c,v retrieving

Fewer pool_get() in kqueue_register()

2020-08-18 Thread Martin Pieuchot
Diff below changes the order of operations in kqueue_register() to get rid of an unnecessary pool_get(). When an event is already present on the list try to acquire it first. Note that knote_acquire() may sleep in which case the list might have changed so the lookup has to always begin from the

kqueue_scan_setup/finish

2020-08-14 Thread Martin Pieuchot
The previous change introducing the kqueue_scan_setup()/finish() API required to switch poll(2) internals to use the kqueue mechanism has been backed out. The reason for the regression is still unknown, so let's take a baby step approach. Diff below introduces the new API with only minimal

Re: TCP congestion control progression

2020-08-14 Thread Martin Pieuchot
On 13/08/20(Thu) 10:14, Brian Brombacher wrote: > > > >> On Aug 9, 2020, at 6:29 PM, Chris Cappuccio wrote: > > Brian Brombacher [br...@planetunix.net] wrote: > >> > >> I am wondering what approach the project is planning to use to modernize > >> the congestion control algorithms. I'm

Re: pppx(4): move ifnet out of KERNEL_LOCK()

2020-08-06 Thread Martin Pieuchot
On 05/08/20(Wed) 12:50, Vitaliy Makkoveev wrote: > pipex(4) and pppx(4) are ready to became a little bit more MP capable. > Diff below moves pppx(4) related `ifnet' out of KERNEL_LOCK(). Nice, one comment below. > Index: sys/net/if_pppx.c >

Re: pipex(4): kill pipexintr()

2020-07-31 Thread Martin Pieuchot
On 31/07/20(Fri) 21:58, Vitaliy Makkoveev wrote: > [...] > What denies us to move pipex(4) under it's own lock? Such question won't lead us anywhere. It assumes it makes sense to move pipex under its own lock. This assumption has many drawback which clearly haven't been studied and more

Re: pipex(4): kill pipexintr()

2020-07-31 Thread Martin Pieuchot
On 31/07/20(Fri) 12:15, Vitaliy Makkoveev wrote: > On Fri, Jul 31, 2020 at 09:36:32AM +0900, YASUOKA Masahiko wrote: > > On Thu, 30 Jul 2020 22:43:10 +0300 > > Vitaliy Makkoveev wrote: > > > On Thu, Jul 30, 2020 at 10:05:13PM +0900, YASUOKA Masahiko wrote: > > >> On Thu, 30 Jul 2020 15:34:09

Re: usbd_abort_pipe(); usbd_close_pipe; dance

2020-07-31 Thread Martin Pieuchot
On 31/07/20(Fri) 11:22, Marcus Glocker wrote: > Maybe I'm missing something here. You aren't. Historically usbd_close_pipe() wasn't aborting transfers. We changed it to do so as it happened to be the easiest fix to some issues that had been copy/pasted. It's just that nobody took the time to do

Re: pipex(4): kill pipexintr()

2020-07-31 Thread Martin Pieuchot
On 30/07/20(Thu) 21:13, YASUOKA Masahiko wrote: > sys/net/if_ethersubr.c: > 372 void > 373 ether_input(struct ifnet *ifp, struct mbuf *m) > (snip) > 519 #if NPPPOE > 0 || defined(PIPEX) > 520 case ETHERTYPE_PPPOEDISC: > 521 case ETHERTYPE_PPPOE: > 522 if (m->m_flags

Re: pipex(4): kill pipexintr()

2020-07-30 Thread Martin Pieuchot
On 29/07/20(Wed) 23:04, Vitaliy Makkoveev wrote: > Now pipex(4) is fully covered by NET_LOCK() and this is documented. But > we still have an issue with pipex(4) session itself and I guess it's > time to fix it. > > We have `pipexinq' and `pipexoutq' mbuf(9) queues to store mbufs. Each > mbuf(9)

Re: xhci(4) isoc: fix bogus handling of chained TRBs

2020-07-28 Thread Martin Pieuchot
On 26/07/20(Sun) 16:23, Marcus Glocker wrote: > On Sun, 26 Jul 2020 13:27:34 + > sc.dy...@gmail.com wrote: > > > On 2020/07/26 10:54, Marcus Glocker wrote: > > > On Sat, 25 Jul 2020 20:31:44 + > > > sc.dy...@gmail.com wrote: > > > > > >> On 2020/07/25 18:10, Marcus Glocker wrote: > >

Re: net80211: skip input block ack window gaps faster

2020-07-28 Thread Martin Pieuchot
On 17/07/20(Fri) 18:15, Stefan Sperling wrote: > On Fri, Jul 17, 2020 at 03:59:38PM +0200, Stefan Sperling wrote: > > While measuring Tx performance at a fixed Tx rate with iwm(4) I observed > > unexpected dips in throughput measured by tcpbench. These dips coincided > > with one or more gap

Re: pipex(4): document global data locks

2020-07-28 Thread Martin Pieuchot
On 17/07/20(Fri) 17:04, Vitaliy Makkoveev wrote: > Subj. Also add NET_ASSERT_LOCKED() to pipex_{link,unlink,rele}_session() > to be sure they called under NET_LOCK(). pipex_rele_session() is freeing memory. When this function is called those chunks of memory shouldn't be referenced by any other

Re: pipex_iface_fini() release multicast session under NET_LOCK()

2020-07-28 Thread Martin Pieuchot
On 17/07/20(Fri) 16:29, Vitaliy Makkoveev wrote: > We are going to lock the whole pipex(4) by NET_LOCK(). So move > `multicast_session' freeing undet NET_LOCK() too. pipex_iface_fini() should be called on the last reference of the descriptor. So this shouldn't be necessary. If there's an issue

Re: fix races in if_clone_create()

2020-07-13 Thread Martin Pieuchot
On 06/07/20(Mon) 15:44, Vitaliy Makkoveev wrote: > > On 6 Jul 2020, at 12:17, Martin Pieuchot wrote: > > Assertions and documentation are more important than preventing races > > because they allow to build awareness and elegant solutions instead of > > hacking diffs

Re: pppac(4): fix races in pppacopen()

2020-07-13 Thread Martin Pieuchot
On 11/07/20(Sat) 23:51, Vitaliy Makkoveev wrote: > [...] > The way you suggest to go is to introduce rwlock and serialize > pppacopen() and pppacclose(). This is bad idea because we will sleep > while we are holding rwlock. That's the whole point of a rwlock to be able to sleep while holding the

Re: pppac(4): fix races in pppacopen()

2020-07-11 Thread Martin Pieuchot
On 10/07/20(Fri) 14:38, Vitaliy Makkoveev wrote: > On Fri, Jul 10, 2020 at 01:22:40PM +0200, Martin Pieuchot wrote: > > On 10/07/20(Fri) 14:07, Vitaliy Makkoveev wrote: > > > We have some races in pppac(4) > > > 1. malloc(9) can sleep so we must check `sc' presence aft

Re: pppac(4): fix races in pppacopen()

2020-07-10 Thread Martin Pieuchot
On 10/07/20(Fri) 14:07, Vitaliy Makkoveev wrote: > We have some races in pppac(4) > 1. malloc(9) can sleep so we must check `sc' presence after malloc(9) Makes sense. > 2. we can sleep between `sc' insertion to `sc_entry' list and > `sc_pipex_iface' initialization. Concurrent pppacioctl() can

Re: pipex(4): kill pipexintr()

2020-07-10 Thread Martin Pieuchot
On 07/07/20(Tue) 01:01, Vitaliy Makkoveev wrote: > On Mon, Jul 06, 2020 at 08:47:23PM +0200, Martin Pieuchot wrote: > > On 06/07/20(Mon) 19:23, Vitaliy Makkoveev wrote: > > > > On 6 Jul 2020, at 17:36, Martin Pieuchot wrote: > > > [...] > > > Unfortu

Re: pppx_if_output() don't lock `pppx_devs_lk'

2020-07-10 Thread Martin Pieuchot
On 08/07/20(Wed) 12:05, Vitaliy Makkoveev wrote: > `pppx_devs_lk' used to protect `pxd_entry' list. We lock `pppx_devs_lk' > in pppx_if_output() to be sure `pxd' is not destroyed by concurrent > pppxclose() but it's useless. We destroy all corresponding `pxi' before > `pxd' and `ifnet's are

Re: USB3 stack with async. transfers support

2020-07-08 Thread Martin Pieuchot
On 07/07/20(Tue) 11:13, Martin wrote: > Hi tech@, > > Not so long ago I've ported UHD driver to support Ettus USRP devices which > uses libusb and asynchronous USB3 data transfers. > Is USB3 async. data stack implemented or planned to have some devices like > USRP working? I'm not aware of

Re: pipex(4): kill pipexintr()

2020-07-06 Thread Martin Pieuchot
On 06/07/20(Mon) 19:23, Vitaliy Makkoveev wrote: > > On 6 Jul 2020, at 17:36, Martin Pieuchot wrote: > [...] > Unfortunately you can’t be sure about NET_LOCK() status while you are > in pppac_start(). It was described at this thread [1]. > > We have two cases: > 1. p

Re: pipex(4): kill pipexintr()

2020-07-06 Thread Martin Pieuchot
On 06/07/20(Mon) 16:42, Vitaliy Makkoveev wrote: > [...] > pipex(4) is simultaneously locked by NET_LOCK() and KERNEL_LOCK() but > with two exceptions: > > 1. As you pointed pipex_pppoe_input() called without KERNEL_LOCK() held. > 2. pppac_start() called without NET_LOCK() held. Or with

Re: fix races in if_clone_create()

2020-07-06 Thread Martin Pieuchot
On 01/07/20(Wed) 00:02, Vitaliy Makkoveev wrote: > On Tue, Jun 30, 2020 at 03:48:22PM +0300, Vitaliy Makkoveev wrote: > > On Tue, Jun 30, 2020 at 12:08:03PM +0200, Martin Pieuchot wrote: > > > On 29/06/20(Mon) 11:59, Vitaliy Makkoveev wrote: > > > > [...] > &g

Re: pipex(4): kill pipexintr()

2020-07-06 Thread Martin Pieuchot
On 01/07/20(Wed) 22:42, Vitaliy Makkoveev wrote: > pipex(4) has 2 mbuf queues: `pipexinq' and `pipexoutq'. When mbuf passed > to pipex it goes to one of these queues and pipexintr() will be > scheduled to process them. pipexintr() called from `netisr' context. > > It's true for pppac(4) but for

Re: fix races in if_clone_create()

2020-06-30 Thread Martin Pieuchot
On 29/06/20(Mon) 11:59, Vitaliy Makkoveev wrote: > [...] > I reworked tool for reproduce. Now I avoided fork()/exec() route and it > takes couple of minutes to take panic on 4 cores. Also some screenshots > attached. Setting kern.pool_debug=2 makes the race reproducible in seconds. Could you

Re: route add ::/0 ...

2020-06-29 Thread Martin Pieuchot
On 28/06/20(Sun) 20:41, YASUOKA Masahiko wrote: > Hi, > > When "::/0" is used as "default", > > # route add ::/0 fe80::1%em0 > add net ::/0: gateway fe80::1%em0: Invalid argument > > route command trims the sockaddr to { .len = 2, .family = AF_INET6 } > for "::/0", but rtable_satoplen()

Re: pipex(4): use reference counters for `ifnet'

2020-06-28 Thread Martin Pieuchot
On 27/06/20(Sat) 17:58, Vitaliy Makkoveev wrote: > > [...] > > Look at r1.329 of net/if.c. Prior to this change if_detach_queues() was > > used to free all mbufs when an interface was removed. Now lazy freeing > > is used everytime if_get(9) rerturns NULL. > > > > This is possible because we

Re: pipex(4): use reference counters for `ifnet'

2020-06-27 Thread Martin Pieuchot
On 27/06/20(Sat) 01:02, Vitaliy Makkoveev wrote: > On Fri, Jun 26, 2020 at 09:15:38PM +0200, Martin Pieuchot wrote: > > On 26/06/20(Fri) 17:53, Vitaliy Makkoveev wrote: > > > On Fri, Jun 26, 2020 at 02:29:03PM +0200, Martin Pieuchot wrote: > > > > On 26/06/20(Fri)

Re: fix races in if_clone_create()

2020-06-27 Thread Martin Pieuchot
On 27/06/20(Sat) 00:35, Vitaliy Makkoveev wrote: > On Fri, Jun 26, 2020 at 09:12:16PM +0200, Martin Pieuchot wrote: > > On 26/06/20(Fri) 16:56, Vitaliy Makkoveev wrote: > > > if_clone_create() has the races caused by context switch. > > > > Can you share a backt

Re: pipex(4): use reference counters for `ifnet'

2020-06-26 Thread Martin Pieuchot
On 26/06/20(Fri) 17:53, Vitaliy Makkoveev wrote: > On Fri, Jun 26, 2020 at 02:29:03PM +0200, Martin Pieuchot wrote: > > On 26/06/20(Fri) 12:35, Vitaliy Makkoveev wrote: > > > On Fri, Jun 26, 2020 at 10:23:42AM +0200, Martin Pieuchot wrote: > > > > On 25/06/20(Thu)

Re: fix races in if_clone_create()

2020-06-26 Thread Martin Pieuchot
On 26/06/20(Fri) 16:56, Vitaliy Makkoveev wrote: > if_clone_create() has the races caused by context switch. Can you share a backtrace of such race? Where does the kernel panic?

Re: make btrace interval event to reciprocal of ticks

2020-06-26 Thread Martin Pieuchot
On 25/06/20(Thu) 23:45, Yuichiro NAITO wrote: > From: Martin Pieuchot > Subject: Re: make btrace interval event to reciprocal of ticks > Date: Thu, 25 Jun 2020 11:36:48 +0200 > > > On 23/06/20(Tue) 12:04, Yuichiro NAITO wrote: > >> Current btrace has `interval:hz:1` pr

Re: pipex(4): use reference counters for `ifnet'

2020-06-26 Thread Martin Pieuchot
On 26/06/20(Fri) 12:35, Vitaliy Makkoveev wrote: > On Fri, Jun 26, 2020 at 10:23:42AM +0200, Martin Pieuchot wrote: > > On 25/06/20(Thu) 19:56, Vitaliy Makkoveev wrote: > > > Updated diff. > > > > > > OpenBSD uses 16 bit counter for allocate interface in

Re: pipex(4): use reference counters for `ifnet'

2020-06-26 Thread Martin Pieuchot
On 25/06/20(Thu) 19:56, Vitaliy Makkoveev wrote: > Updated diff. > > OpenBSD uses 16 bit counter for allocate interface indexes. So we can't > store index in session and be sure if_get(9) returned `ifnet' is our > original `ifnet'. Why not? The point of if_get(9) is to be sure. If that

Re: [PATCH] net: prevent if_clone_destroy from racing with rest of stack

2020-06-25 Thread Martin Pieuchot
On 23/06/20(Tue) 16:21, Martin Pieuchot wrote: > On 23/06/20(Tue) 04:53, Jason A. Donenfeld wrote: > > On 6/23/20, Martin Pieuchot wrote: > > > On 23/06/20(Tue) 01:00, Jason A. Donenfeld wrote: > > >> You can crash a system by running something like: > > &g

Re: make btrace interval event to reciprocal of ticks

2020-06-25 Thread Martin Pieuchot
On 23/06/20(Tue) 12:04, Yuichiro NAITO wrote: > Current btrace has `interval:hz:1` probe that makes periodically events. > `interval:hz:1` looks to make events once per second (= 1Hz), > but current implementation makes once per tick (= 100Hz on amd64). > I think the interval should be counted by

Re: pipex(4): use reference counters for `ifnet'

2020-06-25 Thread Martin Pieuchot
On 24/06/20(Wed) 17:10, Vitaliy Makkoveev wrote: > While `mbuf' enqueued to `pipexinq' or `pipexoutq' it has the reference > to corresponding pipex(4) session as `ph_cookie'. `ph_cookie' is > accessed by pipexintr() and it's always defferent context from context > where we destroy session.

Re: [PATCH] net: prevent if_clone_destroy from racing with rest of stack

2020-06-25 Thread Martin Pieuchot
On 24/06/20(Wed) 19:54, Jason A. Donenfeld wrote: > On Wed, Jun 24, 2020 at 4:02 AM Martin Pieuchot wrote: > > Yes, that might be a better way. If I understood your original mail the > > issue is related to ifunit(), right? ifunit() is not used in packet- > > processing co

Re: [PATCH] net: prevent if_clone_destroy from racing with rest of stack

2020-06-24 Thread Martin Pieuchot
On 23/06/20(Tue) 17:21, Jason A. Donenfeld wrote: > On Tue, Jun 23, 2020 at 8:21 AM Martin Pieuchot wrote: > > I'd argue this is a related problem but a different one. The diff I > > sent serializes cloning/destroying pseudo-interfaces. It has value on > > its own be

Re: xhci(4): acknowledge interrupts before calling usb_schedsoftintr()

2020-06-24 Thread Martin Pieuchot
On 23/06/20(Tue) 23:13, Patrick Wildt wrote: > Hi, > > I had issues with a machine hanging on powerdown. The issue is caused > by sd(4)'s suspend method trying to "power down" my umass(4) USB stick. > > The symptom was that during powerdown, when running in "polling mode", > the first

Re: [PATCH] net: prevent if_clone_destroy from racing with rest of stack

2020-06-23 Thread Martin Pieuchot
On 23/06/20(Tue) 04:53, Jason A. Donenfeld wrote: > On 6/23/20, Martin Pieuchot wrote: > > On 23/06/20(Tue) 01:00, Jason A. Donenfeld wrote: > >> You can crash a system by running something like: > >> > >> for i in 1 2 3; do while true; do ifconfig b

Re: [PATCH] net: prevent if_clone_destroy from racing with rest of stack

2020-06-23 Thread Martin Pieuchot
On 23/06/20(Tue) 01:00, Jason A. Donenfeld wrote: > You can crash a system by running something like: > > for i in 1 2 3; do while true; do ifconfig bridge0 create& ifconfig > bridge0 destroy& done& done > > This works with every type of interface I've tried. It appears that >

Make *select(2) use kqfilter handlers

2020-06-23 Thread Martin Pieuchot
Diff below can be seen as 3 logical parts that together change the current *select(2) implementation: - Create & destroy a per-thread kqueue in fork1() and exit1(). - Change the kqueue_scan() interface to keep track of the end point of a scan, this is mostly from visa@. - Change

Re: New EVFILT_EXCEPT for POLLPRI & POLLRDBAND

2020-06-18 Thread Martin Pieuchot
On 18/06/20(Thu) 09:03, Martin Pieuchot wrote: > On 17/06/20(Wed) 11:50, Martin Pieuchot wrote: > > On 16/06/20(Tue) 06:18, Todd C. Miller wrote: > > > On Tue, 16 Jun 2020 12:48:58 +0200, Martin Pieuchot wrote: > > > > > > > The diff below implements Dr

Re: New EVFILT_EXCEPT for POLLPRI & POLLRDBAND

2020-06-18 Thread Martin Pieuchot
On 17/06/20(Wed) 11:50, Martin Pieuchot wrote: > On 16/06/20(Tue) 06:18, Todd C. Miller wrote: > > On Tue, 16 Jun 2020 12:48:58 +0200, Martin Pieuchot wrote: > > > > > The diff below implements DragonFly's approach of adding a new kind of > > > filter, EVFILT_

Re: New EVFILT_EXCEPT for POLLPRI & POLLRDBAND

2020-06-17 Thread Martin Pieuchot
On 16/06/20(Tue) 06:18, Todd C. Miller wrote: > On Tue, 16 Jun 2020 12:48:58 +0200, Martin Pieuchot wrote: > > > The diff below implements DragonFly's approach of adding a new kind of > > filter, EVFILT_EXCEPT, to report such conditions. This extends the > > existin

Re: New EVFILT_EXCEPT for POLLPRI & POLLRDBAND

2020-06-16 Thread Martin Pieuchot
On 16/06/20(Tue) 06:18, Todd C. Miller wrote: > On Tue, 16 Jun 2020 12:48:58 +0200, Martin Pieuchot wrote: > > > The diff below implements DragonFly's approach of adding a new kind of > > filter, EVFILT_EXCEPT, to report such conditions. This extends the > > existin

New EVFILT_EXCEPT for POLLPRI & POLLRDBAND

2020-06-16 Thread Martin Pieuchot
The kqueue subsystem has no mechanism to indicate "exceptional conditions", that is the equivalent of poll(2)'s POLLPRI & POLLRDBAND and select(2)'s `exceptfds'. The diff below implements DragonFly's approach of adding a new kind of filter, EVFILT_EXCEPT, to report such conditions. This extends

Extend filt_dead() with __EV_HUP

2020-06-15 Thread Martin Pieuchot
This extends the existing dead filter to add __EV_HUP in order to make it usable by deadfs. ok? diff 6125a47a01eb7846ac29f04db756ccc8201dee58 ba22f59fc655d5ccdb16c30de07bb3f10a45aa74 blob - bb1340da842353113b6bed8e4af1427d033d594f blob + 56f58fcabe6ac056e1d8846640ceafedda53707a ---

Re: __EV_HUP to emulate POLLHUP

2020-06-15 Thread Martin Pieuchot
On 13/06/20(Sat) 05:54, Todd C. Miller wrote: > On Sat, 13 Jun 2020 10:05:19 +0200, Martin Pieuchot wrote: > > > Diff below extends the existing kqfilter handlers to be able to set > > POLLHUP in the new poll(2) implementation. > > > > __EV_HUP is introduced and no

__EV_HUP to emulate POLLHUP

2020-06-13 Thread Martin Pieuchot
Diff below extends the existing kqfilter handlers to be able to set POLLHUP in the new poll(2) implementation. __EV_HUP is introduced and now set for this purpose. A new kqfilter for deadfs is introduced to match the existing poll handler. __EV_HUP is not exactly like EV_EOF. Very few

Re: code style questions

2020-06-10 Thread Martin Pieuchot
On 09/06/20(Tue) 20:19, jo...@armadilloaerospace.com wrote: > Looking for some guidance to avoid proposing any unpopular diffs. > > Style(9) says not to use static on file-local functions in the > kernel, because it interferes with the debugger. They still show up > on some functions today; is

Re: Block device poll(2) vs kqueue(2)

2020-06-10 Thread Martin Pieuchot
On 09/06/20(Tue) 11:26, Mark Kettenis wrote: > > Date: Tue, 9 Jun 2020 11:04:02 +0200 > > From: Martin Pieuchot > > > > On 20/05/20(Wed) 14:22, Martin Pieuchot wrote: > > > On 20/05/20(Wed) 12:28, Mark Kettenis wrote: > > > > > Date: Wed, 2

Re: Block device poll(2) vs kqueue(2)

2020-06-09 Thread Martin Pieuchot
On 20/05/20(Wed) 14:22, Martin Pieuchot wrote: > On 20/05/20(Wed) 12:28, Mark Kettenis wrote: > > > Date: Wed, 20 May 2020 11:42:32 +0200 > > > From: Martin Pieuchot > > > > > > Diff below fixes an incoherency between poll(2) and kqueue(2) when it > >

Re: pppx(4): rework to be clese to pseudo-interfaces

2020-06-09 Thread Martin Pieuchot
On 08/06/20(Mon) 15:09, Vitaliy Makkoveev wrote: > On Mon, Jun 08, 2020 at 12:49:05PM +0300, Vitaliy Makkoveev wrote: > > [...] > > There is another way to rewrite pppx_add_session() and > > pipex_add_session(). We can split pipex_add_session() to two > > functions: pipex_init_session() and

Re: pppx(4): rework to be clese to pseudo-interfaces

2020-06-08 Thread Martin Pieuchot
On 29/05/20(Fri) 13:22, Vitaliy Makkoveev wrote: > This time pppx_add_session() has mixed initialisation order. It starts > to initialise pipex(4) session, then initialises `ifnet', then links > pipex(4) session, then continue to initialize `ifnet'. > pppx_add_session() can sleep and

Re: NFS kqueue handlers & poll(2)/select(2) compatibility

2020-06-03 Thread Martin Pieuchot
On 01/06/20(Mon) 15:41, Visa Hankala wrote: > On Sun, May 31, 2020 at 10:48:52AM +0200, Martin Pieuchot wrote: > > NFS poll(2)/select(2) and kqueue(2) behaviors are incoherent. Diff > > below uses the kernel-only NOTE_IMM hint to make the kqueue handlers > > behave like the

NFS kqueue handlers & poll(2)/select(2) compatibility

2020-05-31 Thread Martin Pieuchot
NFS poll(2)/select(2) and kqueue(2) behaviors are incoherent. Diff below uses the kernel-only NOTE_IMM hint to make the kqueue handlers behave like the current poll handler: the poller is bypassed. The new EVFILT_WRITE handler doesn't check for NOTE_IMM because it is unlikely to introduce

FS poll(2)/select(2) behavior with kqfilter handlers

2020-05-31 Thread Martin Pieuchot
FS poll handlers always return true kqueue handlers are different. So the diff below introduces a new NOTE_IMM hint to be able to match the existing poll(2)/select(2) behavior. This hint is obviously kernel-only. This is related to the NFS poller discussion so I'm bringing the diff now in order

Re: Kill NFS-only kqueue poller thread

2020-05-31 Thread Martin Pieuchot
On 30/05/20(Sat) 15:49, Visa Hankala wrote: > [...] > Local filesystems can observe changes at the source, which makes polling > unnecessary. NFS clients do not have that benefit. The NFSv3 protocol > lacks a mechanism to notify clients of changes. > > The NFS polling mechanism is in use for

Re: Kill NFS-only kqueue poller thread

2020-05-31 Thread Martin Pieuchot
On 30/05/20(Sat) 09:25, Theo de Raadt wrote: > [...] > What does this have to do with threads? That is an implimentation detail. This implementation detail is specific to NFS, no other FS do anything like that. So I'm questioning whether calling kthread_create(9) inside a kqueue(2) handler,

Re: Kill NFS-only kqueue poller thread

2020-05-30 Thread Martin Pieuchot
On 30/05/20(Sat) 09:22, Visa Hankala wrote: > On Thu, May 28, 2020 at 12:11:20PM +0200, Martin Pieuchot wrote: > > When it comes to kqueue filters NFS is special. A custom thread is > > created when the first event is registered. Its purpose is to poll > > for changes ever

Re: pipex(4): kill NET_LOCK() in pipex_ioctl()

2020-05-29 Thread Martin Pieuchot
On 28/05/20(Thu) 15:27, Vitaliy Makkoveev wrote: > On Thu, May 28, 2020 at 12:26:39PM +0200, Martin Pieuchot wrote: > > On 27/05/20(Wed) 11:54, Vitaliy Makkoveev wrote: > > > pipex(4) is simultaneously protected by KERNEL_LOCK() and NET_LOCK() and > > > the last is not

Re: unlock PF_UNIX sockets

2020-05-29 Thread Martin Pieuchot
On 28/05/20(Thu) 14:59, Vitaliy Makkoveev wrote: > socket(2) layer is already protected by solock(). It grabs NET_LOCK() > for inet{,6}(4) sockets, but all other sockets are still under > KERNEL_LOCK(). > > I guess solock is already placed everythere it's required. Also `struct > file' is already

Re: WireGuard patchset for OpenBSD, rev. 2

2020-05-28 Thread Martin Pieuchot
On 27/05/20(Wed) 20:18, Matt Dunwoodie wrote: > On Wed, 27 May 2020 09:34:53 +0200 > Martin Pieuchot wrote: > > Regarding the kernel, I'd suggest you use "#if NWG > 0" like it is > > done for other pseudo-drives with 'needs-flag'. > > For the most part the

Re: pipex(4): kill NET_LOCK() in pipex_ioctl()

2020-05-28 Thread Martin Pieuchot
On 27/05/20(Wed) 11:54, Vitaliy Makkoveev wrote: > pipex(4) is simultaneously protected by KERNEL_LOCK() and NET_LOCK() and > the last is not required. I guess to start remove NET_LOCK(). Diff below > drops NET_LOCK() in pipex_ioctl() and underlaying routines. At least > this helps to kill

Re: Initialize v4l2_requestbuffers struct to avoid invalid mmap

2020-05-28 Thread Martin Pieuchot
On 26/05/20(Tue) 11:30, Ingo Feinerer wrote: > video(1) supports reading frames from a webcam via mmap(). To inform the > V4L2 device about the number of desired buffers containing the frames to > be memory-mapped, a VIDIOC_REQBUFS ioctl call is used. > > At the moment the v4l2_requestbuffers

Re: pppx(4): prevent access to `pxi' being destroyed

2020-05-28 Thread Martin Pieuchot
On 26/05/20(Tue) 16:12, Vitaliy Makkoveev wrote: > `pppx_if' has `pxi_ready' field used to prevent access to incomplete > `pxi'. But we don't prevent access to `pxi' which we destroy. > pppx_if_destroy() can sleep so we can grab `pxi' which we already > destroying by concurrent thread and cause

Kill NFS-only kqueue poller thread

2020-05-28 Thread Martin Pieuchot
When it comes to kqueue filters NFS is special. A custom thread is created when the first event is registered. Its purpose is to poll for changes every 2.5sec. This logic has been inherited from NetBSD and is not present in FreeBSD. Since filt_nfsread() only check `n_size' of a given nfsnode

Remove KERNEL_LOCK() in socket(2) and socketpair(2)

2020-05-27 Thread Martin Pieuchot
When these two syscalls have been marked NOLOCK, falloc(), fdinsert() & friends weren't ready to be executed without KERNEL_LOCK(). This is no longer true. kqueue(2), for example, do the same dances without this lock. ok? Index: kern/uipc_syscalls.c

Re: WireGuard patchset for OpenBSD, rev. 2

2020-05-27 Thread Martin Pieuchot
Hello Matt, Thank you for your submission. On 26/05/20(Tue) 19:39, Matt Dunwoodie wrote: > After some feedback and comments, we've addressed the concerns, and > fixed a few things from our side too. Overall the structure is familiar > with no major changes, so any prior readings mostly carry

Re: [RFC] pppd: add pipex(4) L2TP control support

2020-05-27 Thread Martin Pieuchot
On 26/05/20(Tue) 10:31, Claudio Jeker wrote: > [...] > npppd(8) is server only it can not establish a connection. pppd(8) on the > other hand is more client side (but I think it can do both). Could someone knowledgable indicate that in the man pages? Currently there is: npppd – new

Re: [RFC] pppd: add pipex(4) L2TP control support

2020-05-26 Thread Martin Pieuchot
On 25/05/20(Mon) 21:42, Sergey Ryazanov wrote: > Add dedicated option to activate kernel L2TP acceleration via > the pipex(4). The options should be passed by a L2TP tunnel > management daemon (e.g. xl2tpd). What is the difference between npppd(8) and pppd(8)? Aren't those two redundant? Why

Re: kqueue_scan() refactoring

2020-05-25 Thread Martin Pieuchot
On 10/04/20(Fri) 14:43, Martin Pieuchot wrote: > Diff below reduces kqueue_scan() to the collect of events on a given > kqueue and let its caller, sys_kevent(), responsible for the copyout(9). > > Apart from the code simplification, this refactoring clearly separates > kqu

Introduce kqueue_terminate() & kqueue_free()

2020-05-25 Thread Martin Pieuchot
Small refactoring needed to manage per-thread kqueues. Such kqueue are not associated to a file descriptor, that's why the functions below take a "struct kqueue *" as argument. ok? Index: kern/kern_event.c === RCS file:

Re: vmd(8) and thread safety: a quick proof of concept using libevent 2.1 from ports

2020-05-25 Thread Martin Pieuchot
On 25/05/20(Mon) 10:14, Claudio Jeker wrote: > [...] > One approach to fix this is to ensure that the event functions are > exclusivly used in the main event thread but as usual it is easy to > introduce bugs again. This is a common issue with multi-threaded applications using non thread-safe

Re: vmd(8) and thread safety: a quick proof of concept using libevent 2.1 from ports

2020-05-25 Thread Martin Pieuchot
On 24/05/20(Sun) 07:56, Dave Voutila wrote: > On Sat, May 23, 2020 at 9:38 PM Dave Voutila wrote: > > > > Hello tech@, > > > > Attached is a diff that patches vmd(8) to utilize libevent 2.1 (from > > ports) in an attempt to test the hypothesis that thread safety will > > help stabilize Linux

  1   2   3   4   5   6   7   8   9   10   >