Re: syslogd sendmsg

2021-09-11 Thread Vitaliy Makkoveev
> On 11 Sep 2021, at 21:16, Martijn van Duren > wrote: > > On Sat, 2021-09-11 at 04:05 +0300, Vitaliy Makkoveev wrote: >> On Fri, Sep 10, 2021 at 07:40:58PM +0200, Alexander Bluhm wrote: >>> Hi, >>> >>> After converting syslogd messages to iov, U

Re: syslogd sendmsg

2021-09-11 Thread Vitaliy Makkoveev
> On 11 Sep 2021, at 21:51, Alexander Bluhm wrote: > > On Sat, Sep 11, 2021 at 04:05:31AM +0300, Vitaliy Makkoveev wrote: >> On Fri, Sep 10, 2021 at 07:40:58PM +0200, Alexander Bluhm wrote: >>> Hi, >>> >>> After converting syslogd messages to iov, UDP

Re: ipsec: remove unused `PolicyHead' from 'sockaddr_encap' structure

2021-07-13 Thread Vitaliy Makkoveev
On Mon, Jul 12, 2021 at 01:47:25PM +0200, Tobias Heider wrote: > On Sun, Jul 11, 2021 at 05:33:18AM +0300, Vitaliy Makkoveev wrote: > > This member is never set or used. Also I kept 'SENT_IP6' definition for > > prevent the potential break of third party software. Is it ok

iked(8): Increase the default Child SA data lifetime limit

2021-08-02 Thread Vitaliy Makkoveev
iked(8) uses 3 hours and 512 megabytes of processed data as default lifetime hard limits for Child SA. Also it sets 85-95% of these values as soft limit. iked(8) should perform rekeying before we reach hard limit otherwise this SA will be killed and the tunnel stopped. With default values the

Re: iked(8): Increase the default Child SA data lifetime limit

2021-08-02 Thread Vitaliy Makkoveev
> On 3 Aug 2021, at 04:22, Theo de Raadt wrote: > > Joerg Sonnenberger wrote: > >> On Tue, Aug 03, 2021 at 01:12:54AM +0300, Vitaliy Makkoveev wrote: >>> Index: sbin/iked/types.h >>> === >&g

Re: iked(8): Increase the default Child SA data lifetime limit

2021-08-03 Thread Vitaliy Makkoveev
On Tue, Aug 03, 2021 at 01:40:51PM +0200, Tobias Heider wrote: > On Tue, Aug 03, 2021 at 12:17:38PM +0100, Stuart Henderson wrote: > > On 2021/08/03 01:12, Vitaliy Makkoveev wrote: > > > iked(8) uses 3 hours and 512 megabytes of processed data as default > > > lifeti

Re: iked(8): Increase the default Child SA data lifetime limit

2021-08-03 Thread Vitaliy Makkoveev
On Tue, Aug 03, 2021 at 12:17:38PM +0100, Stuart Henderson wrote: > On 2021/08/03 01:12, Vitaliy Makkoveev wrote: > > iked(8) uses 3 hours and 512 megabytes of processed data as default > > lifetime hard limits for Child SA. Also it sets 85-95% of these values as > > soft

Re: iked(8): Increase the default Child SA data lifetime limit

2021-08-03 Thread Vitaliy Makkoveev
On Mon, Aug 02, 2021 at 09:09:03PM -0600, Theo de Raadt wrote: > > I suspect the first step is to make the rekey decision be based upon the > strength of the ciphers. > Do you mean the special default limits for each cipher?

unp_externalize() and `unp_lock'

2021-10-12 Thread Vitaliy Makkoveev
I want to move a little bit forward with fine grained locking in UNIX domain sockets area. My goal is to introduce the new rwlock(9) for garbage collector data and keep existing `unp_lock' only for per-socket data. This will allow to replace it by per-socket `so_lock' in the future. I like to

Re: unp_internalize() and `unp_lock'

2021-10-21 Thread Vitaliy Makkoveev
A little update for previous diff: `unp_lock' is taken around the whole fd_getfile(9) loop. Index: sys/kern/uipc_usrreq.c === RCS file: /cvs/src/sys/kern/uipc_usrreq.c,v retrieving revision 1.149 diff -u -p -r1.149 uipc_usrreq.c ---

Re: ipsec tdb error handling

2021-10-21 Thread Vitaliy Makkoveev
On Thu, Oct 21, 2021 at 12:08:57PM +0200, Alexander Bluhm wrote: > On Sat, Oct 16, 2021 at 12:50:46AM +0300, Vitaliy Makkoveev wrote: > > This time tdb_delete() doesn't kill passed `tdb' but schedules timeout > > to do this. So this `tdb' is logically killed and should be not used,

Re: crypto: remove crypto queue

2021-10-21 Thread Vitaliy Makkoveev
On Fri, Oct 22, 2021 at 12:03:07AM +0200, Tobias Heider wrote: > Currently, all crypto users set CRYPTO_F_NOQUEUE to run crypto operations > without queue and there are no plans to switch back to using the queue. > The diff below removes the flag together with the queueing code. > > ok? > ok

Re: ipsec tdb error handling

2021-10-15 Thread Vitaliy Makkoveev
Hi, This time tdb_delete() doesn't kill passed `tdb' but schedules timeout to do this. So this `tdb' is logically killed and should be not used, but because the killer timeout is also serialized with netlock we don;t catch use-after-free issue. This was the reason I reverted my "per-cpu

unp_internalize() and `unp_lock'

2021-10-15 Thread Vitaliy Makkoveev
The next diff before introduce rwlock(9) for UNIX sockets garbage collector data. Release `unp_lock' before call unp_internalize() and take it within when access garbage collector data such as `unp_rights', `unp_msgcount' and `unp_file'. The garbage collector rwlock(9) will be introduced with the

Re: unp_internalize() and `unp_lock'

2021-10-15 Thread Vitaliy Makkoveev
On Fri, Oct 15, 2021 at 01:32:22PM +0200, Mark Kettenis wrote: > > Date: Fri, 15 Oct 2021 14:04:08 +0300 > > From: Vitaliy Makkoveev > > > > The next diff before introduce rwlock(9) for UNIX sockets garbage > > collector data. > > > > Release `unp_lo

Re: rtfree(): "rt->rt_refcnt > 0" assertion

2021-10-02 Thread Vitaliy Makkoveev
On Sat, Oct 02, 2021 at 11:27:48AM +0200, Martin Pieuchot wrote: > On 15/09/21(Wed) 01:23, Vitaliy Makkoveev wrote: > > We have weird `rt_refcnt' check in rtfree(): > > > > > 497 rtfree(struct rtentry *rt) > > 498 { > > ... > > 504 refcnt = (

Re: more ipsec mbuf pointer

2021-10-24 Thread Vitaliy Makkoveev
> On 24 Oct 2021, at 19:00, Alexander Bluhm wrote: > > On Sun, Oct 24, 2021 at 02:52:45AM +0200, Alexander Bluhm wrote: >> There are more m_pullup()s in IPsec input. Pass down the pointer >> to the mbuf. At the end it will reach ip_deliver() which expects >> a pointer to an mbuf anyway. > >

Re: ipsec tdb error handling

2021-10-22 Thread Vitaliy Makkoveev
> On 22 Oct 2021, at 18:23, Alexander Bluhm wrote: > > On Thu, Oct 21, 2021 at 03:04:22PM +0300, Vitaliy Makkoveev wrote: >> With this diff the `tdb' dereference after tdb_free() is safe enough and >> I have no objections against your diff. > > My diff did not ma

Re: tdb_delete_locked for pfkey

2021-12-03 Thread Vitaliy Makkoveev
ok mvs@ > On 3 Dec 2021, at 20:25, Tobias Heider wrote: > > Hi, > > the diff below adds tdb_delete_locked() for use in pfkeyv2_sa_flush(). > This way we won't have to worry about keeping the inline code and > tdb_delete() in sync. > > ok? > > Index: net/pfkeyv2.c >

Re: parallel ip forwarding

2021-12-03 Thread Vitaliy Makkoveev
On Fri, Dec 03, 2021 at 08:35:45PM +0100, Alexander Bluhm wrote: > Hi, > > After implementing MP safe refcounting for IPsec TDB, I wonder how > stable my diff for forwarding on multiple CPU is. > > Note that IPsec still has the workaround to disable multiple queues. > We do not have a mutex that

Re: Rework UNIX sockets locking to be fine grained

2021-12-06 Thread Vitaliy Makkoveev
Updated on top of latest source. Index: sys/kern/uipc_socket.c === RCS file: /cvs/src/sys/kern/uipc_socket.c,v retrieving revision 1.269 diff -u -p -r1.269 uipc_socket.c --- sys/kern/uipc_socket.c 11 Nov 2021 16:35:09 -

Re: ipsec tdb walk

2021-12-19 Thread Vitaliy Makkoveev
On Sat, Dec 18, 2021 at 01:07:00AM +0100, Alexander Bluhm wrote: > Hi, > > There are occasions where the walker in tdb_walk() might sleep. > Case SADB_DUMP is such a case. And mvs@ has a diff that sleeps to > read the counters. So holding the tdb_sadb_mtx() when calling > walker() is not

Re: unlock mmap(2) for anonymous mappings

2021-12-31 Thread Vitaliy Makkoveev
The uvm_wxabort path within uvm_wxcheck() looks not MP-safe. > On 31 Dec 2021, at 12:14, Klemens Nanni wrote: > > Now that mpi has unlocked uvm's fault handler, we can unlock the mmap > syscall to handle MAP_ANON without the big lock. > > sys_mmap() still protects the !MAP_ANON case, i.e. file

Re: parallel ip forwarding

2021-12-24 Thread Vitaliy Makkoveev
`rtt_link’ list also has missing protection when rtfree() called outside netlock. > On 25 Dec 2021, at 01:34, Alexander Bluhm wrote: > > On Fri, Dec 24, 2021 at 02:04:17PM +0100, Alexander Bluhm wrote: >> On Fri, Dec 24, 2021 at 12:55:04AM +0100, Alexander Bluhm wrote: >>> If you use only

Re: ip_deliver without kernel lock

2021-12-24 Thread Vitaliy Makkoveev
ok mvs@ > On 25 Dec 2021, at 01:17, Alexander Bluhm wrote: > > Hi, > > ip_deliver() has been called without kernel lock from ip_ours() and > ip6_ours() for a long time. It looks like these two callers in ip6 > input were forgotten to be unlocked. > > ok? > > bluhm > > Index:

Re: parallel ip forwarding

2021-12-25 Thread Vitaliy Makkoveev
On Fri, Dec 24, 2021 at 12:50:23PM +0100, Alexander Bluhm wrote: > On Fri, Dec 24, 2021 at 04:16:28PM +0900, YASUOKA Masahiko wrote: > > > - npppd l2pt ipsecflowinfo is not MP safe > > > > Does this mean the things we are discussing on the "Fix > > ipsp_spd_lookup() for transport mode" thread? I

Re: parallel ip forwarding

2021-12-30 Thread Vitaliy Makkoveev
> On 30 Dec 2021, at 11:28, YASUOKA Masahiko wrote: > > Hi, > > On Sat, 25 Dec 2021 21:50:47 +0300 > Vitaliy Makkoveev wrote: >> On Fri, Dec 24, 2021 at 12:50:23PM +0100, Alexander Bluhm wrote: >>> On Fri, Dec 24, 2021 at 04:16:28PM +0900, YASUOKA M

Unlock getpeername(2)

2022-01-03 Thread Vitaliy Makkoveev
Subj. The getpeername(2) sysckall is pretty simple. For inet and unix sockets it follows the code which was already unlocked with accept(2) unlocking. Just copy the 'sockaddr' structure containing the peer address. For key management and route domain sockets it just copies the read-only global

Re: IPsec tdb ref counting

2021-11-15 Thread Vitaliy Makkoveev
On Sun, Nov 14, 2021 at 10:50:34PM +0100, Alexander Bluhm wrote: > On Sat, Nov 13, 2021 at 06:04:07PM +0100, Alexander Bluhm wrote: > > It passes regress but there are setups that are not covered. Bridge > > and pfsync with IPsec and TCP signature need special care. > > > > When testing, please

Re: Rework UNIX sockets locking to be fine grained

2021-11-20 Thread Vitaliy Makkoveev
On Sat, Nov 20, 2021 at 07:16:40PM +0100, Matthias Schmidt wrote: > Hi Vitaliy, > > * Vitaliy Makkoveev wrote: > > Updated diff. Re-lock dances were simplified in the unix(4) sockets > > layer. > > I am running this diff and the one before on a Thinkpad T450s which i

Re: IPsec tdb ref counting

2021-11-23 Thread Vitaliy Makkoveev
On Tue, Nov 23, 2021 at 06:54:59AM +0100, Hrvoje Popovski wrote: > On 21.11.2021. 23:36, Alexander Bluhm wrote: > > Updated tdb refcounting diff after merging with mvs@'s commit. > > Hi, > > after 24 hours hitting sasyncd setup one box panic > > r620-2# panic: pool_do_get: tdb free list

Expose TDBs with excess hardlimit in ipsec(4) statistics

2021-11-20 Thread Vitaliy Makkoveev
When I had playing with bluhm@'s ipsec(4) diffs I found my system always manages to complete rekeying and I have no TDB's with hardlimit excess. It seems I'm not the only such person and other testers also don't have panics Hrvoje Popovski had. I want to count the TDBs with hardlimit excess as

Re: IPsec tdb ref counting

2021-11-20 Thread Vitaliy Makkoveev
On Sat, Nov 20, 2021 at 05:31:33PM +0100, Alexander Bluhm wrote: > New tdb refcounting diff. > > I delete and unref the timeouts earlier and fixed some leaks. At > least on my machine it does not crash and tcp InUse is 0 after > ipsecctl -F . > > Please test. > > mvs: Are some of your

Re: Expose TDBs with excess hardlimit in ipsec(4) statistics

2021-11-20 Thread Vitaliy Makkoveev
> On 21 Nov 2021, at 04:14, Alexander Bluhm wrote: > > On Sun, Nov 21, 2021 at 02:56:58AM +0300, Vitaliy Makkoveev wrote: >> Also I found the `ipsec_notdb' counter description in header doesn't >> math to netstat(1) description. We never count `ipsec_notdb' and the >

Re: IPsec tdb ref counting

2021-11-19 Thread Vitaliy Makkoveev
I want to note all the people who testing this diff. Please be sure your test exceeds the lifetime of the `tdb’ and some rekeying cycles have been made. > On 19 Nov 2021, at 20:09, Alexander Bluhm wrote: > > Hi, > > I got a new hardware in my testlab and could panic this machine > easily with

Re: UNIX sockets: use vnode(9) lock to protect `v_socket' dereference

2021-11-11 Thread Vitaliy Makkoveev
On Thu, Nov 11, 2021 at 03:21:45PM +0100, Alexander Bluhm wrote: > On Thu, Nov 11, 2021 at 12:09:41PM +0300, Vitaliy Makkoveev wrote: > > Can I propose to commit this diff before? It reorders soclose() to > > destroy PCB before `so_q0' and `so_q' cleanup. > > OK bluhm@

Re: ipsec ip deliver

2021-11-10 Thread Vitaliy Makkoveev
On Tue, Nov 09, 2021 at 05:27:16PM +0100, Alexander Bluhm wrote: > On Tue, Nov 09, 2021 at 02:00:01PM +0100, Denis Fondras wrote: > > > @@ -352,19 +343,19 @@ ipsec_common_input(struct mbuf **mp, int > > >* Call appropriate transform and return -- callback takes care of > > >* everything

Re: UNIX sockets: use vnode(9) lock to protect `v_socket' dereference

2021-11-10 Thread Vitaliy Makkoveev
Ping... On Thu, Nov 04, 2021 at 05:30:07PM +0300, Vitaliy Makkoveev wrote: > On Fri, Nov 05, 2021 at 08:31:07AM +0100, Martin Pieuchot wrote: > > On 26/10/21(Tue) 14:12, Vitaliy Makkoveev wrote: > > > Another step to make UNIX sockets locking fine grained. > > > >

Re: UNIX sockets: use vnode(9) lock to protect `v_socket' dereference

2021-11-10 Thread Vitaliy Makkoveev
On Wed, Nov 10, 2021 at 10:03:48PM +0100, Alexander Bluhm wrote: > On Tue, Oct 26, 2021 at 02:12:36PM +0300, Vitaliy Makkoveev wrote: > > --- sys/kern/uipc_socket.c 14 Oct 2021 23:05:10 - 1.265 > > +++ sys/kern/uipc_socket.c 26 Oct 2021 11:05:59 - > > @@ -31

UNIX sockets: move garbage collector data out from `unp_lock'

2021-11-11 Thread Vitaliy Makkoveev
The final step before rework UNIX sockets to fine grained locks. Except `unp_ino' this leaves only per-socket data protected by `unp_lock'. The `unp_ino' protection is not the big deal and will be done with mutex(9) in the future diff. The `unp_gc_lock' rwlock(9) protects `unp_defer',

Re: UNIX sockets: move garbage collector data out from `unp_lock'

2021-11-13 Thread Vitaliy Makkoveev
On Fri, Nov 12, 2021 at 03:28:42AM +0300, Vitaliy Makkoveev wrote: > The final step before rework UNIX sockets to fine grained locks. Except > `unp_ino' this leaves only per-socket data protected by `unp_lock'. The > `unp_ino' protection is not the big deal and will be done wit

Re: IPsec tdb ref counting

2021-11-13 Thread Vitaliy Makkoveev
Hi, Do you have panics with this diff? Index: sys/net/if_bridge.c === RCS file: /cvs/src/sys/net/if_bridge.c,v retrieving revision 1.358 diff -u -p -r1.358 if_bridge.c --- sys/net/if_bridge.c 11 Nov 2021 18:08:17 - 1.358

Re: IPsec tdb ref counting

2021-11-13 Thread Vitaliy Makkoveev
On Sat, Nov 13, 2021 at 09:49:31PM +, Stuart Henderson wrote: > On 2021/11/13 18:04, Alexander Bluhm wrote: > > Hi, > > > > To make IPsec MP safe we need refcounting for the tdb. The diff > > below is part of something bigger we have at genua. Although it > > does not cover timeouts and the

Re: IPsec tdb ddb print

2021-11-16 Thread Vitaliy Makkoveev
On Mon, Nov 15, 2021 at 05:23:43PM +0100, Alexander Bluhm wrote: > Hi, > > To debug IPsec and tdb refcounting it may be useful to have "show > tdb" and "show all tdbs" in ddb. > > ok? > Indeed this is useful. ok mvs@ > bluhm > > Index: share/man/man4/ddb.4 >

Re: IPsec tdb ref counting

2021-11-24 Thread Vitaliy Makkoveev
On Wed, Nov 24, 2021 at 02:12:15PM +0100, Alexander Bluhm wrote: > On Wed, Nov 24, 2021 at 02:48:06AM +0300, Vitaliy Makkoveev wrote: > > >> ddb{2}> show panic > > >> *cpu2: pool_do_get: tdb free list modified: page 0x8801; > > >> item ad

Re: Rework UNIX sockets locking to be fine grained

2021-11-24 Thread Vitaliy Makkoveev
On Wed, Nov 24, 2021 at 11:36:22AM +0100, Martin Pieuchot wrote: > On 22/11/21(Mon) 14:42, Vitaliy Makkoveev wrote: > > On Sat, Nov 20, 2021 at 03:12:31AM +0300, Vitaliy Makkoveev wrote: > > > Updated diff. Re-lock dances were simplified in the unix(4) sockets > > > la

Re: IPsec tdb ref counting

2021-11-24 Thread Vitaliy Makkoveev
It was not clean for me because we has no extra reference for parallel tdb_delete(). Now I understand how it should work and why ‘TDBF_DELETED’ fixed this. Thanks for explanation. > On 24 Nov 2021, at 17:52, Alexander Bluhm wrote: > > On Wed, Nov 24, 2021 at 05:12:36PM +0300, Vitaliy

Re: IPsec tdb ref counting

2021-11-24 Thread Vitaliy Makkoveev
> On 24 Nov 2021, at 22:14, Tobias Heider wrote: > > On Wed, Nov 24, 2021 at 03:52:26PM +0100, Alexander Bluhm wrote: >> On Wed, Nov 24, 2021 at 05:12:36PM +0300, Vitaliy Makkoveev wrote: >>> Understood. But his means we encoded double unref when we calling >>>

Re: IPsec tdb ref counting

2021-11-25 Thread Vitaliy Makkoveev
On Thu, Nov 25, 2021 at 09:52:54AM +0100, Alexander Bluhm wrote: > On Sat, Nov 13, 2021 at 06:04:07PM +0100, Alexander Bluhm wrote: > > When testing, please check for tdb leaks. > > The diff below was running on my performance setup for more than > 10 hours. iked SA lifetime was about 10

Re: Rework UNIX sockets locking to be fine grained

2021-11-22 Thread Vitaliy Makkoveev
On Sat, Nov 20, 2021 at 03:12:31AM +0300, Vitaliy Makkoveev wrote: > Updated diff. Re-lock dances were simplified in the unix(4) sockets > layer. > > Reference counters added to unix(4) sockets layer too. This makes > pointer dereference of peer's control block always saf

Unlock accept(2) and accept4(2) syscalls

2021-11-22 Thread Vitaliy Makkoveev
Since the rev1.267 of kern/uipc_socket solock() is used as klist lock for sockets to make socket event filters MP-safe. This means KNOTE(9) within doaccept() doesn't require kernel lock to be held and the accept(2) and accept4(2) syscalls could be unlocked. This makes sense because all our sockets

Re: Rework UNIX sockets locking to be fine grained

2021-11-19 Thread Vitaliy Makkoveev
Updated diff. Re-lock dances were simplified in the unix(4) sockets layer. Reference counters added to unix(4) sockets layer too. This makes pointer dereference of peer's control block always safe after re-lock. The `unp_refs' list cleanup done in the unp_detach(). This removes the case where

Re: UNIX sockets: move garbage collector data out from `unp_lock'

2021-11-15 Thread Vitaliy Makkoveev
ping... On Sat, Nov 13, 2021 at 07:20:23PM +0300, Vitaliy Makkoveev wrote: > On Fri, Nov 12, 2021 at 03:28:42AM +0300, Vitaliy Makkoveev wrote: > > The final step before rework UNIX sockets to fine grained locks. Except > > `unp_ino' this leaves only per-socket data protect

Re: IPsec tdb ref counting

2021-11-15 Thread Vitaliy Makkoveev
On Mon, Nov 15, 2021 at 12:35:13PM +0100, Hrvoje Popovski wrote: > On 14.11.2021. 22:50, Alexander Bluhm wrote: > > New diff with fix from mvs@. Please continue testing with this one. > > Hi, > > i've applied this diff on sasyncd setup with two ipsec sessions and i'm > getting this panic. Box

Re: UNIX sockets: use vnode(9) lock to protect `v_socket' dereference

2021-11-11 Thread Vitaliy Makkoveev
On Thu, Nov 11, 2021 at 12:42:36AM +0300, Vitaliy Makkoveev wrote: > On Wed, Nov 10, 2021 at 10:03:48PM +0100, Alexander Bluhm wrote: > > On Tue, Oct 26, 2021 at 02:12:36PM +0300, Vitaliy Makkoveev wrote: > > > --- sys/kern/uipc_socket.c14 Oct 2021 23:05:10 - 1

Re: IPsec tdb ref counting

2021-11-15 Thread Vitaliy Makkoveev
On Mon, Nov 15, 2021 at 02:51:16PM +0100, Hrvoje Popovski wrote: And you don'n see "> tdb_free() killing ourself" in dmesg output? > On 15.11.2021. 13:11, Vitaliy Makkoveev wrote: > > Hi, > > > > Could you try this diff? It should

Rework UNIX sockets locking to be fine grained

2021-11-18 Thread Vitaliy Makkoveev
The UNIX sockets garbage collector was moved out of global `unp_lock' rwlock(9) which locks the whole layer. Now it's used to protect per-socket data and it's time to replace it to per-socket's `so_lock'. Unlike PF_ROUTE and PF_KEY sockets we have the paths where multiple sockets should be locked

Re: IPsec tdb ref counting

2021-11-23 Thread Vitaliy Makkoveev
> On 23 Nov 2021, at 18:16, Tobias Heider wrote: > > On Tue, Nov 23, 2021 at 02:18:26PM +0100, Alexander Bluhm wrote: >> On Tue, Nov 23, 2021 at 06:54:59AM +0100, Hrvoje Popovski wrote: >>> after 24 hours hitting sasyncd setup one box panic >> >> Thanks for testing. >> >> I have reduced my

Re: ipsec: refactor TDBF_DELETED

2021-11-25 Thread Vitaliy Makkoveev
On Thu, Nov 25, 2021 at 10:59:25PM +0100, Alexander Bluhm wrote: > On Thu, Nov 25, 2021 at 05:13:16PM +0100, Tobias Heider wrote: > > Now with the missing parts from pfkeyv2.c as noticed by Hrvoje. > > We have this code in pfkeyv2_send() > > if

Re: ipsec: refactor TDBF_DELETED

2021-11-25 Thread Vitaliy Makkoveev
ok mvs@ > On 26 Nov 2021, at 01:37, Tobias Heider wrote: > > On Fri, Nov 26, 2021 at 01:17:22AM +0300, Vitaliy Makkoveev wrote: >> On Thu, Nov 25, 2021 at 10:59:25PM +0100, Alexander Bluhm wrote: >>> On Thu, Nov 25, 2021 at 05:13:16PM +0100, Tobias Heider wrote: >>

UNIX sockets: make `unp_rights', `unp_msgcount' and `unp_file' atomic

2021-10-30 Thread Vitaliy Makkoveev
This completely removes global rwlock(9) from the unp_internalize() and unp_externalize() normal paths but only leaves it in unp_externalize() error path. Also we don't need to simultaneously hold both fdplock() and `unp_lock' in unp_internalize(). As non obvious profit this simplifies the future

UNIX sockets: use vnode(9) lock to protect `v_socket' dereference

2021-10-26 Thread Vitaliy Makkoveev
Another step to make UNIX sockets locking fine grained. The listening socket has the references from file descriptors layer and from the vnode(9) layer. This means when we close(2)'ing such socket it still referenced by concurrent thread through connect(2) path. When we bind(2) UNIX socket we

soqinsque(): replace 'DIAGNOSTIC' block by KASSERT(9)

2021-10-26 Thread Vitaliy Makkoveev
Just for consistency with the rest of kern/uipc_socket2.c Index: sys/kern/uipc_socket2.c === RCS file: /cvs/src/sys/kern/uipc_socket2.c,v retrieving revision 1.113 diff -u -p -r1.113 uipc_socket2.c --- sys/kern/uipc_socket2.c 26

Re: UNIX sockets: use vnode(9) lock to protect `v_socket' dereference

2021-11-05 Thread Vitaliy Makkoveev
On Fri, Nov 05, 2021 at 08:31:07AM +0100, Martin Pieuchot wrote: > On 26/10/21(Tue) 14:12, Vitaliy Makkoveev wrote: > > Another step to make UNIX sockets locking fine grained. > > > > The listening socket has the references from file descriptors layer and > > from the

Re: UNIX sockets: make `unp_rights', `unp_msgcount' and `unp_file' atomic

2021-11-05 Thread Vitaliy Makkoveev
On Fri, Nov 05, 2021 at 09:50:05AM +0100, Mark Kettenis wrote: > > Date: Fri, 5 Nov 2021 07:18:03 +0100 > > From: Martin Pieuchot > > > > On 30/10/21(Sat) 21:22, Vitaliy Makkoveev wrote: > > > This completely removes global rwlock(9) from the unp_internalize() a

Re: crypto: retire async crypto API

2021-10-23 Thread Vitaliy Makkoveev
> On 23 Oct 2021, at 16:08, Tobias Heider wrote: > > Now that we have removed all the legacy crypto offloading drivers we can > simplify the crypto framework API by ripping out the async callbacks. > > The diff below removes crypto_dispatch() and crypto_done() and replaces > them with

Re: ipsec input mbuf pointer

2021-10-23 Thread Vitaliy Makkoveev
> On 22 Oct 2021, at 21:38, Alexander Bluhm wrote: > > Hi, > > I found a m_pullup() down in ah input. As it may free or change > the mbuf, the caller must be careful. All callers do not use the > mbuf, so we are safe. Nevertheless I would like to use a common > pattern to handle this. Pass

Re: ipsp_spd_inp in in_pcbconnect

2021-10-25 Thread Vitaliy Makkoveev
> On 26 Oct 2021, at 00:52, Alexander Bluhm wrote: > > Hi, > > The implementation of ipsp_spd_inp() is side effect free. It sets > the error output parameter and returns a tdb. Both are ignored in > in_pcbconnect(). So this code does nothing. > > ok? > ok mvs@ > bluhm > > Index:

Re: UNIX sockets: use vnode(9) lock to protect `v_socket' dereference

2021-10-28 Thread Vitaliy Makkoveev
ping... > On 26 Oct 2021, at 14:12, Vitaliy Makkoveev wrote: > > Another step to make UNIX sockets locking fine grained. > > The listening socket has the references from file descriptors layer and > from the vnode(9) layer. This means when we close(2)'ing such socket it

Re: Rework UNIX sockets locking to be fine grained

2021-12-01 Thread Vitaliy Makkoveev
On Wed, Dec 01, 2021 at 12:55:06PM +0100, Martin Pieuchot wrote: > On 24/11/21(Wed) 15:04, Vitaliy Makkoveev wrote: > > [...] > > Really, this is the simplest way. The shared lock for the pair of > > sockets moves re-lock dances to the connect and disconnect stages whi

Re: ipsp_spd_lookup error

2021-12-01 Thread Vitaliy Makkoveev
On Wed, Dec 01, 2021 at 12:18:30AM +0100, Alexander Bluhm wrote: > Hi, > > I want to ref count the TDB that is returned by ipsp_spd_lookup(). > > Sometimes the TDB returned by ipsp_spd_lookup() is not used. So > refcounting that, does not make sense. As a first step I want to > convert

Re: Rework UNIX sockets locking to be fine grained

2021-12-02 Thread Vitaliy Makkoveev
Thanks! Bulk build make us sure all packages have been built with new headers. > On 2 Dec 2021, at 16:26, Christian Weisgerber wrote: > > Vitaliy Makkoveev: > >> Include missing "sys/refcnt.h" header to unpcb.h to fix libkvm and >> netstat(1) build. No funct

Re: Rework UNIX sockets locking to be fine grained

2021-12-01 Thread Vitaliy Makkoveev
This diff included into the snaps. > On 30 Nov 2021, at 19:48, Vitaliy Makkoveev wrote: > > Include missing "sys/refcnt.h" header to unpcb.h to fix libkvm and > netstat(1) build. No functional changes. > > I

Re: Fix ipsp_spd_lookup() for transport mode

2021-11-30 Thread Vitaliy Makkoveev
Hi, This question is mostly for bluhm@. Should the gettdbbyflow() grab the extra reference on returned `tdbp' like other other gettdb*() do? I'm pointing this because we are going to not rely on the netlock when doing `tdbp' dereference. Also could this block be rewritten? It looks a little

Re: IPsec TDB locking

2021-12-07 Thread Vitaliy Makkoveev
[IN] looks strange. If this field modified after creation it is mutable. There are cases when such field could modified only once, but it still is atomic. We have cases where we do assignment only once, like `unp_addr’ when we bind(2)ing socket and we don’t modify if until socket’s destruction.

Re: IPsec TDB locking

2021-12-08 Thread Vitaliy Makkoveev
ok mvs@ > On 8 Dec 2021, at 16:41, Alexander Bluhm wrote: > > On Wed, Dec 08, 2021 at 02:41:47AM +0300, Vitaliy Makkoveev wrote: >> [IN] looks strange. If this field modified after creation it is >> mutable. There are cases when such field could modified only once, >

Re: IPsec TDB locking

2021-12-08 Thread Vitaliy Makkoveev
, at 08:56, Visa Hankala wrote: > > On Wed, Dec 08, 2021 at 02:41:47AM +0300, Vitaliy Makkoveev wrote: >> [IN] looks strange. If this field modified after creation it is >> mutable. There are cases when such field could modified only once, >> but it still is atomic. >> >

Re: IPsec delete TDB in ipo cache

2021-12-07 Thread Vitaliy Makkoveev
> On 7 Dec 2021, at 18:31, Alexander Bluhm wrote: > > Hi, > > In ipo_tdb the flow contains a reference counted TDB cache. This > may prevent that tdb_free() is called. It is not a real leak as > ipsecctl -F or termination of iked flush this cache. The kernel > does the cleanup itself if we

Re: Rework UNIX sockets locking to be fine grained

2021-11-30 Thread Vitaliy Makkoveev
Include missing "sys/refcnt.h" header to unpcb.h to fix libkvm and netstat(1) build. No functional changes. Index: sys/kern/uipc_socket.c === RCS file: /cvs/src/sys/kern/uipc_socket.c,v retrieving revision 1.269 diff -u -p -r1.269

Re: ipsec ipo tdb mutex

2021-12-14 Thread Vitaliy Makkoveev
ok mvs@ > On 11 Dec 2021, at 22:03, Alexander Bluhm wrote: > > On Sat, Dec 11, 2021 at 12:53:35AM +0100, Alexander Bluhm wrote: >> To cache lookups, the policy ipo is linked to its SA tdb. There >> is a list of SAs that belong to a policy. To make it MP safe we >> need a mutex around these

Re: syskaller igmp leavegroup

2021-12-14 Thread Vitaliy Makkoveev
ok mvs@ > On 14 Dec 2021, at 16:49, Alexander Bluhm wrote: > > Hi, > > syzkaller found a NULL dereference: > > https://syzkaller.appspot.com/bug?id=a7f159c677ec125fe9edef2265e2749f13e24243 > > It looks like inm->inm_rti is NULL. It is set in rti_fill() or not > set if malloc(9) fails.

Per-cpu counters for TDB stats

2021-12-15 Thread Vitaliy Makkoveev
The previous version of this diff exposed UAF issue we had after tdb_delete(). bluhm@ fixed this and proposes to put per-cpu counters diff again to tree. This is the updated diff to be against the resent sources. Index: sys/net/pfkeyv2_convert.c

Re: Per-cpu counters for TDB stats

2021-12-15 Thread Vitaliy Makkoveev
Please drop this diff. > On 15 Dec 2021, at 19:21, Vitaliy Makkoveev wrote: > > The previous version of this diff exposed UAF issue we had after > tdb_delete(). bluhm@ fixed this and proposes to put per-cpu counters > diff again to tree. This is the updated diff to be aga

Re: parallel ipsec workaround

2021-07-20 Thread Vitaliy Makkoveev
On Tue, Jul 20, 2021 at 03:41:32PM +0200, Alexander Bluhm wrote: > Hi, > > The current workaround to disable parallel IPsec does not work. > Variable nettaskqs must not change at runtime. Interface input > queues choose the thread during init with ifiq_softnet = net_tq(). > So it cannot be

Re: crypto error ipsec counter

2021-07-20 Thread Vitaliy Makkoveev
ok mvs@ > On 21 Jul 2021, at 02:13, Alexander Bluhm wrote: > > Hi, > > Propagate the crypto errors and count them in ipsec. This is part > of a larger diff where I disable the crypto queues for ipsec. I > think it cannot happen, but errors should always be checked. > > tq is never NULL. >

Re: ipsec: document locks of some structures

2021-07-18 Thread Vitaliy Makkoveev
ping? The diff below updated to the most recent source. Index: sys/netinet/ip_ipsp.h === RCS file: /cvs/src/sys/netinet/ip_ipsp.h,v retrieving revision 1.203 diff -u -p -r1.203 ip_ipsp.h --- sys/netinet/ip_ipsp.h 18 Jul 2021

Re: forwarding in parallel ipsec workaround

2021-07-19 Thread Vitaliy Makkoveev
On Mon, Jul 19, 2021 at 05:53:40PM +0200, Alexander Bluhm wrote: > Hi, > > I found why the IPsec workaround did not work. > > At init time we set ifiq->ifiq_softnet = net_tq(ifp->if_index + > idx), but the workaround modifies net_tq() at runtime. Modifying > net_tq() at runtime is bad anyway as

Re: forwarding in parallel

2021-07-19 Thread Vitaliy Makkoveev
On Mon, Jul 19, 2021 at 06:40:07PM +0200, Alexander Bluhm wrote: > On Fri, Jul 09, 2021 at 10:47:49PM +0300, Vitaliy Makkoveev wrote: > > If I understood your diff right, pipex(4) is also affected through: > > > > ip_local() > >-> ip_deliver() > > ->

Re: forwarding in parallel

2021-07-19 Thread Vitaliy Makkoveev
On Mon, Jul 19, 2021 at 10:56:32PM +0200, Alexander Bluhm wrote: > On Mon, Jul 19, 2021 at 08:02:30PM +0300, Vitaliy Makkoveev wrote: > > I mean the case when ip_local() called by ip_ours(). Unfortunately, I'm > > not familiar with PPTP but it looks affected because it don't use

pipex(4): use per-CPU counters for session statistics

2021-07-19 Thread Vitaliy Makkoveev
With bluhm@'s "forwarding in parallel" diff pipex(4) session could be accessed in parallel through (*ifp->if_input) -> ether_input(). Except statistics and MPPE data PPPOE sessions are mostly immutable. We have only place where we should reset 'idle_time' in input path with shared netlock. But

Re: forwarding in parallel ipsec workaround

2021-07-21 Thread Vitaliy Makkoveev
On Wed, Jul 21, 2021 at 06:41:30PM +0200, Alexander Bluhm wrote: > On Mon, Jul 19, 2021 at 07:33:55PM +0300, Vitaliy Makkoveev wrote: > > Hi, pipex(4) is also not ready for parallel access. In the chunk below > > it will be accessed through (*ifp->if_input

pipex(4): introduce mutexes to protect 'pipex_session' context

2021-07-21 Thread Vitaliy Makkoveev
With bluhm@'s diff for parallel forwarding pipex(4) could be accessed in parallel through (*ifp->if_input)() -> ether_input() -> pipex_pppoe_input(). PPPOE pipex(4) sessions are mostly immutable except MPPE crypt. The diff below makes pipex(4) a little bit more parallelization reliable. The new

Re: forwarding in parallel ipsec workaround

2021-07-22 Thread Vitaliy Makkoveev
On Thu, Jul 22, 2021 at 08:38:04PM +0200, Hrvoje Popovski wrote: > On 22.7.2021. 12:21, Hrvoje Popovski wrote: > > Thank you for explanation.. > > > > after hitting box all night, box panic and i was able to reproduce it > > this morning ... I'm not sure but box panic after hour or more of > >

ipsec(4): use per-CPU counters for tunnel descriptor block (tdb) statistics

2021-07-22 Thread Vitaliy Makkoveev
This makes 'tdb' struct more MP compliant. 'tdb_data' struct became unused and was removed. Index: sys/net/pfkeyv2_convert.c === RCS file: /cvs/src/sys/net/pfkeyv2_convert.c,v retrieving revision 1.72 diff -u -p -r1.72

Re: forwarding in parallel ipsec workaround

2021-07-22 Thread Vitaliy Makkoveev
On Thu, Jul 22, 2021 at 12:21:47PM +0200, Hrvoje Popovski wrote: > On 22.7.2021. 0:39, Alexander Bluhm wrote: > > On Thu, Jul 22, 2021 at 12:06:09AM +0200, Hrvoje Popovski wrote: > >> I'm combining this and last parallel diff and i can't see any drops in > >> traffic. Even sending at high rate,

Re: forwarding in parallel ipsec workaround

2021-07-23 Thread Vitaliy Makkoveev
On Thu, Jul 22, 2021 at 11:30:02PM +0200, Hrvoje Popovski wrote: > On 22.7.2021. 22:52, Vitaliy Makkoveev wrote: > > On Thu, Jul 22, 2021 at 08:38:04PM +0200, Hrvoje Popovski wrote: > >> On 22.7.2021. 12:21, Hrvoje Popovski wrote: > >>> Thank you for explanation.. >

Re: Kill sbinsertoob()

2021-07-24 Thread Vitaliy Makkoveev
ok mvs@ > On 24 Jul 2021, at 12:26, Martin Pieuchot wrote: > > This function is unused, killing it means we need fewer refactoring to > switch to a per-socket mutex serializing event notifications. > > ok? > > Index: kern/uipc_socket2.c >

Re: pipex(4): introduce mutexes to protect 'pipex_session' context

2021-07-27 Thread Vitaliy Makkoveev
On Tue, Jul 27, 2021 at 02:44:51AM +0200, Alexander Bluhm wrote: > On Tue, Jul 27, 2021 at 12:30:06AM +0300, Vitaliy Makkoveev wrote: > > > The diff below makes pipex(4) a little bit more parallelization > > > reliable. > > > > @@ -2238,10 +2240,6 @@ pipex_m

Re: pipex(4): introduce mutexes to protect 'pipex_session' context

2021-07-26 Thread Vitaliy Makkoveev
ping > On 22 Jul 2021, at 01:56, Vitaliy Makkoveev wrote: > > With bluhm@'s diff for parallel forwarding pipex(4) could be accessed in > parallel through (*ifp->if_input)() -> ether_input() -> > pipex_pppoe_input(). PPPOE pipex(4) sessions are mostly immut

Re: ipsec(4): use per-CPU counters for tunnel descriptor block (tdb) statistics

2021-07-26 Thread Vitaliy Makkoveev
ping > On 23 Jul 2021, at 03:12, Vitaliy Makkoveev wrote: > > This makes 'tdb' struct more MP compliant. 'tdb_data' struct became > unused and was removed. > > Index: sys/net/pfkeyv2_convert.c > === > RC

<    1   2   3   4   5   6   7   8   >