Re: npppd pptp hangs
On Sat, Apr 04, 2020 at 08:22:27PM +0200, Marko Cupać wrote: > On 2020-03-31 10:07, Marko Cupać wrote: > > On Mon, 30 Mar 2020 14:33:46 +0300 > > Vitaliy Makkoveev wrote: > > > > > On Mon, Mar 30, 2020 at 02:28:08PM +0300, Vitaliy Makkoveev wrote: > > > > You have pipex(4) disabled. Is it still hangs with disabled > > > > pipex(4)? As I discovered > > > > (https://marc.info/?t=15852997681=1=2), npppd with pipex(4) > > > > enabled and non-NULL "idle-timeout" option will crash kernel. You > > > > can disable this option in yout npppd.conf an reenable pipex(4). > > > > Looks like crashes should gone. > > > And don't use pppac(4) with pipex enabled, use pppx(4). Crash you > > > reported https://marc.info/?t=15850622592=1=2 is actual for > > > pppac(4). > > > > > > > Thanks for the instruction. > > > > I have: > > - left net.pipex.enable=1 > > - replaced tun1 with pppx0 in npppd.conf > > - removed 'pipex no' from npppd.conf > > > > So far so good, I'll send update if I experience further hangs. > > No crash since changing interface from tun to pppx. > > Thanx! Ok :) > > -- > Before enlightenment - chop wood, draw water. > After enlightenment - chop wood, draw water. > > Marko Cupać > https://www.mimar.rs/ >
Re: npppd pptp hangs
On 2020-03-31 10:07, Marko Cupać wrote: On Mon, 30 Mar 2020 14:33:46 +0300 Vitaliy Makkoveev wrote: On Mon, Mar 30, 2020 at 02:28:08PM +0300, Vitaliy Makkoveev wrote: > You have pipex(4) disabled. Is it still hangs with disabled > pipex(4)? As I discovered > (https://marc.info/?t=15852997681=1=2), npppd with pipex(4) > enabled and non-NULL "idle-timeout" option will crash kernel. You > can disable this option in yout npppd.conf an reenable pipex(4). > Looks like crashes should gone. And don't use pppac(4) with pipex enabled, use pppx(4). Crash you reported https://marc.info/?t=15850622592=1=2 is actual for pppac(4). Thanks for the instruction. I have: - left net.pipex.enable=1 - replaced tun1 with pppx0 in npppd.conf - removed 'pipex no' from npppd.conf So far so good, I'll send update if I experience further hangs. No crash since changing interface from tun to pppx. Thanx! -- Before enlightenment - chop wood, draw water. After enlightenment - chop wood, draw water. Marko Cupać https://www.mimar.rs/
Re: npppd pptp hangs
On Mon, 30 Mar 2020 14:33:46 +0300 Vitaliy Makkoveev wrote: > On Mon, Mar 30, 2020 at 02:28:08PM +0300, Vitaliy Makkoveev wrote: > > You have pipex(4) disabled. Is it still hangs with disabled > > pipex(4)? As I discovered > > (https://marc.info/?t=15852997681=1=2), npppd with pipex(4) > > enabled and non-NULL "idle-timeout" option will crash kernel. You > > can disable this option in yout npppd.conf an reenable pipex(4). > > Looks like crashes should gone. > And don't use pppac(4) with pipex enabled, use pppx(4). Crash you > reported https://marc.info/?t=15850622592=1=2 is actual for > pppac(4). > Thanks for the instruction. I have: - left net.pipex.enable=1 - replaced tun1 with pppx0 in npppd.conf - removed 'pipex no' from npppd.conf So far so good, I'll send update if I experience further hangs. The instruction on pppx(4) you gave me came as surprising news to me. I have been using npppd since it still had undocumented 'old style' config. Once npppd.conf got its manpage (was it 5.3?) I've set up tun1 as PPTP interface and it worked great with up to ~20 clients all these years. I was very satisfied that all PPTP traffic went through single interface (as opposed to my previous setup with poptop which created separate tun interface for each session), as I had the ability to graph its traffic from SNMP data. I guess I was 'holding it wrong' all this time, and yet it worked well :) Thank you once again. -- Before enlightenment - chop wood, draw water. After enlightenment - chop wood, draw water. Marko Cupać https://www.mimar.rs/
Re: npppd pptp hangs
On Mon, Mar 30, 2020 at 02:28:08PM +0300, Vitaliy Makkoveev wrote: > You have pipex(4) disabled. Is it still hangs with disabled pipex(4)? > As I discovered (https://marc.info/?t=15852997681=1=2), npppd > with pipex(4) enabled and non-NULL "idle-timeout" option will crash > kernel. You can disable this option in yout npppd.conf an reenable > pipex(4). Looks like crashes should gone. And don't use pppac(4) with pipex enabled, use pppx(4). Crash you reported https://marc.info/?t=15850622592=1=2 is actual for pppac(4).
Re: npppd pptp hangs
On Mon, Mar 30, 2020 at 12:47:13PM +0200, Marko Cupać wrote:
> On Sat, 28 Mar 2020 01:46:41 +0300
> Vitaliy Makkoveev wrote:
>
> > Can you try latest snapshot?
>
> Unfortunately, the box that runs npppd is the most important machine on
> my network (GRE/IPsec hub for multiple branch offices), I can't take the
> risk.
>
> > Can you share your npppd.conf?
>
> Below, I have redacted sensitive information. Perhaps it is worth
> mentioning that npppd listens on IP address of CARP interface.
>
> ---npppd.conf.start---
> # GLOBAL
> set max-session 200
> set user-max-session 1
>
> # TUNNEL
> tunnel EXAMPLEORG protocol pptp {
> listen on IP.ADD.RE.SS
> pptp-hostname vpn.example.org
> pptp-vendor-name "openbsd-npppd"
> ingress-filter yes
> pipex no
> mppe required
> mppe-key-length 128
> mppe-key-state stateless
> idle-timeout 1800
> }
>
> # IPCP
> ipcp KAPPASTAR {
> pool-address "IP.ADD.RE.SS/24"
> dns-servers IP.ADD.RE.SS
> allow-user-selected-address no
> }
>
> # INTERFACE
> interface tun1 address IP.ADD.RE.SS ipcp EXAMPLEORG
>
> # AUTHENTICATION
> authentication RADIUS type radius {
> strip-nt-domain yes
> strip-atmark-realm yes
> authentication-server {
> address IP.ADD.RE.SS secret "ThisIsNotRealPassword"
> }
> accounting-server {
> address IP.ADD.RE.SS secret "ThisIsNotRealPassword"
> }
> }
>
> bind tunnel from EXAMPLEORG authenticated by RADIUS to tun1
> ---npppd.conf.end---
>
> Thank you in advance for looking into it.
> --
> Before enlightenment - chop wood, draw water.
> After enlightenment - chop wood, draw water.
>
> Marko Cupać
> https://www.mimar.rs/
>
You have pipex(4) disabled. Is it still hangs with disabled pipex(4)?
As I discovered (https://marc.info/?t=15852997681=1=2), npppd
with pipex(4) enabled and non-NULL "idle-timeout" option will crash
kernel. You can disable this option in yout npppd.conf an reenable
pipex(4). Looks like crashes should gone.
Re: npppd pptp hangs
On Sat, 28 Mar 2020 01:46:41 +0300
Vitaliy Makkoveev wrote:
> Can you try latest snapshot?
Unfortunately, the box that runs npppd is the most important machine on
my network (GRE/IPsec hub for multiple branch offices), I can't take the
risk.
> Can you share your npppd.conf?
Below, I have redacted sensitive information. Perhaps it is worth
mentioning that npppd listens on IP address of CARP interface.
---npppd.conf.start---
# GLOBAL
set max-session 200
set user-max-session 1
# TUNNEL
tunnel EXAMPLEORG protocol pptp {
listen on IP.ADD.RE.SS
pptp-hostname vpn.example.org
pptp-vendor-name "openbsd-npppd"
ingress-filter yes
pipex no
mppe required
mppe-key-length 128
mppe-key-state stateless
idle-timeout 1800
}
# IPCP
ipcp KAPPASTAR {
pool-address "IP.ADD.RE.SS/24"
dns-servers IP.ADD.RE.SS
allow-user-selected-address no
}
# INTERFACE
interface tun1 address IP.ADD.RE.SS ipcp EXAMPLEORG
# AUTHENTICATION
authentication RADIUS type radius {
strip-nt-domain yes
strip-atmark-realm yes
authentication-server {
address IP.ADD.RE.SS secret "ThisIsNotRealPassword"
}
accounting-server {
address IP.ADD.RE.SS secret "ThisIsNotRealPassword"
}
}
bind tunnel from EXAMPLEORG authenticated by RADIUS to tun1
---npppd.conf.end---
Thank you in advance for looking into it.
--
Before enlightenment - chop wood, draw water.
After enlightenment - chop wood, draw water.
Marko Cupać
https://www.mimar.rs/
Re: npppd pptp hangs
> On 24 Mar 2020, at 12:09, Marko Cupać wrote: > > On Tue, 24 Mar 2020 09:34:09 +0100 > Marko Cupać wrote: > >> On Tue, 24 Mar 2020 07:13:27 +1000 >> Stuart Longland wrote: >> >>> On 23/3/20 10:26 pm, Marko Cupać wrote: Anything I can do to avoid future hangs? > > I got another hang, this time killing npppd process crashed complete OS > (sorry for photo, I don't have serial console set up): > > https://oblak.mimar.rs/index.php/s/Cc9J745jH93RK6j > > At the time when npppd wouldn't accept new connections, and npppctl > won't return anything, but before the crash, i noticed high CPU usage > in top: > > 45125 _ppp 640 3128K 6340K onproc/3 -39:05 99.85% npppd > > Pehaps bugs@ would be more appropriate list? > > -- > Before enlightenment - chop wood, draw water. > After enlightenment - chop wood, draw water. > > Marko Cupać > https://www.mimar.rs/ > Can you try latest snapshot? Can you share your npppd.conf?
Re: npppd pptp hangs
On Tue, 24 Mar 2020 09:34:09 +0100 Marko Cupać wrote: > On Tue, 24 Mar 2020 07:13:27 +1000 > Stuart Longland wrote: > > > On 23/3/20 10:26 pm, Marko Cupać wrote: > > > Anything I can do to avoid future hangs? I got another hang, this time killing npppd process crashed complete OS (sorry for photo, I don't have serial console set up): https://oblak.mimar.rs/index.php/s/Cc9J745jH93RK6j At the time when npppd wouldn't accept new connections, and npppctl won't return anything, but before the crash, i noticed high CPU usage in top: 45125 _ppp 640 3128K 6340K onproc/3 -39:05 99.85% npppd Pehaps bugs@ would be more appropriate list? -- Before enlightenment - chop wood, draw water. After enlightenment - chop wood, draw water. Marko Cupać https://www.mimar.rs/
Re: npppd pptp hangs
On Tue, 24 Mar 2020 07:13:27 +1000 Stuart Longland wrote: > On 23/3/20 10:26 pm, Marko Cupać wrote: > > Anything I can do to avoid future hangs? > > Whilst probably not the answer you're looking for: moving away from > PPTP would be a good start. > > The MSCHAPv2 authentication used in PPTP is vulnerable to dictionary > attacks and the RC4 cipher used in MPPE (the security layer of PPTP) > is laughably weak in today's security context. Whilst MSCHAPv2 can be > replaced with EAP-TLS, there's no fix for MPPE. > > IPSec (which is built into OpenBSD) or OpenVPN (in ports) would be > vastly superior options. Indeed, I am also waiting for the day when I'll be able to point iked to Microsoft's implementation of a RADIUS server (NPS), which will authenticate Active Directory domain-joined machines by their machine certificate and hopefully with additional domain user password for 2FA, authorise them by Active Directory group membership, and log their accounting in format which can be easily parsed and converted into human-readable statistics with currently available parsers. Uh, that sounded like I'm some kind of Microsoft fanboy, but I'm not. I just have to provide hundreds of Windows users a way to access resources on a corporate network in order to keep my bills paid. npppd's pptp helps me brilliantly (anyone remember poptop? that was hell :) Anyway, I use IPSec extensively to connect branch office routers, both in tunnel mode for passive clients with dynamic IPs, and in transport mode for protecting GRE tunnels (OSPF). Lately I'm adding multipath redundancy over multiple ISPs using rdomains. OpenVPN also has a place on my network. OpenBSD is a miracle :) Pardon my blatant self-promotion on link below, but I think it's a win-win situation - I get eternal fame and glory on the Internet, and list readers get copy/paste howto set up npppd pptp server with RADIUS authentication. Could come handy in this "end of days" situation where everyone works remotely :D https://www.mimar.rs/blog/how-to-set-up-pptp-vpn-server-with-openbsd-and-npppd Best regards, -- Before enlightenment - chop wood, draw water. After enlightenment - chop wood, draw water. Marko Cupać https://www.mimar.rs/
Re: npppd pptp hangs
On 23/3/20 10:26 pm, Marko Cupać wrote: > Anything I can do to avoid future hangs? Whilst probably not the answer you're looking for: moving away from PPTP would be a good start. The MSCHAPv2 authentication used in PPTP is vulnerable to dictionary attacks and the RC4 cipher used in MPPE (the security layer of PPTP) is laughably weak in today's security context. Whilst MSCHAPv2 can be replaced with EAP-TLS, there's no fix for MPPE. IPSec (which is built into OpenBSD) or OpenVPN (in ports) would be vastly superior options. -- Stuart Longland (aka Redhatter, VK4MSL) I haven't lost my mind... ...it's backed up on a tape somewhere.
npppd pptp hangs
Hi, my npppd pptp server has recently got increase from ~20 to >200 concurrent users. So far it worked flawlessly for years, but before few minutes it become unresponsive. It stopped logging at one point (I have log redirected to its own file, /var/log/npppd). npppctl also hanged, returning nothing. I couldn't restart it with rcctl, or kill it with HUP. I had to resort to `kill -9', and it started fine afterwards. It appears that already established sessions worked, but with poor performance. I have lots of these in log (I saw them earlier as well but they weren't causing problems AFAIK): Mar 23 12:03:26 nat1 /bsd: pipex: ppp=1869 iface=tun1 protocol=PPTP id=45012 Received bad data packet: out of sequence: seq=1266880(1266946-1267010) ack=1915237(1915368-1915471) Mar 23 12:03:26 nat1 /bsd: pipex: ppp=1869 iface=tun1 protocol=PPTP id=45012 Received bad data packet: out of sequence: seq=1266881(1266946-1267010) ack=1915239(1915368-1915472) Mar 23 12:03:26 nat1 /bsd: pipex: ppp=1869 iface=tun1 protocol=PPTP id=45012 Workaround the out-of-sequence PPP framing problem: 1215 => 1151 Mar 23 12:06:59 nat1 /bsd: pipex: ppp=1847 iface=tun1 protocol=PPTP id=45439 received packet caused window overflow. seq=218469(218273-218337)may lost 196 packets. Also, at the time before killing it there's: Mar 23 13:13:37 nat1 /bsd: splassert: pipex_destroy_session: want 2 have 0 Mar 23 13:13:37 nat1 last message repeated 95 times Anything I can do to avoid future hangs? Thank you in advance, -- Before enlightenment - chop wood, draw water. After enlightenment - chop wood, draw water. Marko Cupać https://www.mimar.rs/
