On Sat, Jul 10, 2021 at 05:26:57PM +0200, Martin Pieuchot wrote: > One of the reasons for the drop of performances in the kqueue-based > poll/select is the fact that kqueue filters are called up to 3 times > per syscall and that they all spin on the NET_LOCK() for TCP/UDP > packets. > > Diff below is a RFC for improving the situation. > > socket kqueue filters mainly check for the amount of available items to > read/write. This involves comparing various socket buffer fields (sb_cc, > sb_lowat, etc). The diff below introduces a new mutex to serialize > updates of those fields with reads in the kqueue filters. > > Since these fields are always modified with the socket lock held, either > the mutex or the solock are enough to have a coherent view of them. > Note that either of these locks is necessary only if multiple fields > have to be read (like in sbspace()). > > Other per-socket fields accessed in the kqueue filters are never > combined (with &&) to determine a condition. So assuming it is fine to > read register-sized fields w/o the socket lock we can safely remove it > there. > > Could such mutex also be used to serialize klist updates?
I think the lock should be such that it can serialize socket klists. As the main motivator for this change is kqueue, the viability of using the mutex for the klist locking should be checked now. The mutex has to be held whenever calling KNOTE() on sb_sel.si_note, or selwakeup() on sb_sel. Then the socket f_event callbacks will not need to lock the mutex themselves. I had a diff that serialized socket klists using solock(). It did not work well because it increased lock contention, especially when using kqueue as backend for poll(2) and select(2). The diff is not even correct any longer since recent changes to socket locking have introduced new lock order constraints that conflict with it.
