Re: high sofnet load with gif(4) and icmp
On Thu, Apr 05, 2018 at 02:38:36PM +0200, Alexander Bluhm wrote: > OK bluhm@ > > On Thu, Apr 05, 2018 at 09:14:32AM +1000, David Gwynne wrote: > > Index: if_gif.c > > === > > RCS file: /cvs/src/sys/net/if_gif.c,v > > retrieving revision 1.113 > > diff -u -p -r1.113 if_gif.c > > --- if_gif.c15 Mar 2018 21:01:18 - 1.113 > > +++ if_gif.c4 Apr 2018 23:12:02 - > > @@ -403,6 +403,8 @@ gif_output(struct ifnet *ifp, struct mbu > > error = ENOBUFS; > > goto drop; > > } > > + memcpy((caddr_t)(mtag + 1), &ifp->if_index, sizeof(ifp->if_index)); > > + m_tag_prepend(m, mtag); > > > > m->m_pkthdr.ph_family = dst->sa_family; > > Is this considered a regression, which is worth an errata? We have quite a few gif tunnels. Marc
Re: high sofnet load with gif(4) and icmp
Hi, I'm optimistic about this patch: where previously running traffic through this router over gif tunnels would result in memory exhaustion, the problem now seems gone. memory graph: http://instituut.net/~job/screenshots/ee7f0fa5304032a2.png Should perhaps an errata / syspatch blob be prepared for the 6.3 users that rely on gif(4)? I can attempt to create one for amd64. Kind regards, Job On Thu, Apr 5, 2018 at 2:38 PM, Alexander Bluhm wrote: > OK bluhm@ > > On Thu, Apr 05, 2018 at 09:14:32AM +1000, David Gwynne wrote: >> Index: if_gif.c >> === >> RCS file: /cvs/src/sys/net/if_gif.c,v >> retrieving revision 1.113 >> diff -u -p -r1.113 if_gif.c >> --- if_gif.c 15 Mar 2018 21:01:18 - 1.113 >> +++ if_gif.c 4 Apr 2018 23:12:02 - >> @@ -403,6 +403,8 @@ gif_output(struct ifnet *ifp, struct mbu >> error = ENOBUFS; >> goto drop; >> } >> + memcpy((caddr_t)(mtag + 1), &ifp->if_index, sizeof(ifp->if_index)); >> + m_tag_prepend(m, mtag); >> >> m->m_pkthdr.ph_family = dst->sa_family; >> >
Re: high sofnet load with gif(4) and icmp
OK bluhm@ On Thu, Apr 05, 2018 at 09:14:32AM +1000, David Gwynne wrote: > Index: if_gif.c > === > RCS file: /cvs/src/sys/net/if_gif.c,v > retrieving revision 1.113 > diff -u -p -r1.113 if_gif.c > --- if_gif.c 15 Mar 2018 21:01:18 - 1.113 > +++ if_gif.c 4 Apr 2018 23:12:02 - > @@ -403,6 +403,8 @@ gif_output(struct ifnet *ifp, struct mbu > error = ENOBUFS; > goto drop; > } > + memcpy((caddr_t)(mtag + 1), &ifp->if_index, sizeof(ifp->if_index)); > + m_tag_prepend(m, mtag); > > m->m_pkthdr.ph_family = dst->sa_family; >
Re: high sofnet load with gif(4) and icmp
David Gwynne writes: > On Wed, Apr 04, 2018 at 08:27:52PM +0200, Christian Barthel wrote: >> Hello, >> >> After upgrading to 6.3, I've encountered a problem while using gif(4): >> >> Pre-Cond: fresh install of OpenBSD 6.3 >> # cat /etc/hostname.gif0 >> tunnel 10.0.0.1 10.0.0.2 >> 10.0.0.1 10.0.0.2 >> # sh /etc/netstart.sh gif0 >> # ping 10.0.0.2 >> >> >> Result & Behavior: the kernel thread softnet starts running on the cpu >> (STATE: run) and consumes 100% cpu time spent in system mode. The >> memory consumption increases until there is no free memory (mbufpl, >> mtagpl seem to grow). Even upon stopping/suspending the ping(1) process, >> the sofnet thread continues to consume cpu time. Only destroying the >> gif0 interface changes the softnet STATE to sleep (memory seems still >> allocated). >> >> I've discovered this on a Hetzner virtual machine (VPS) System (dmesg >> and sysinfo attached) but i also reproduced the behavior on a Thinkpad >> X200 (dmesg not attached). >> >> Has anyone seen something similar or is there something I am missing? I >> tried to backtrack (at least a bit) and I can reproduce the behavior at >> least up to revision 1.109 (if_gif.c). Not sure if my config is absolute >> correct but in anyway, I guess softnet shouldn't consume so much cpu >> time in that case. > > can you try this diff? Thanks for your reply and having looked at it. I applied the patch and it solved the above described issue. Christian > > Index: if_gif.c > === > RCS file: /cvs/src/sys/net/if_gif.c,v > retrieving revision 1.113 > diff -u -p -r1.113 if_gif.c > --- if_gif.c 15 Mar 2018 21:01:18 - 1.113 > +++ if_gif.c 4 Apr 2018 23:12:02 - > @@ -403,6 +403,8 @@ gif_output(struct ifnet *ifp, struct mbu > error = ENOBUFS; > goto drop; > } > + memcpy((caddr_t)(mtag + 1), &ifp->if_index, sizeof(ifp->if_index)); > + m_tag_prepend(m, mtag); > > m->m_pkthdr.ph_family = dst->sa_family; > >
Re: high sofnet load with gif(4) and icmp
On Wed, Apr 04, 2018 at 08:27:52PM +0200, Christian Barthel wrote: > Hello, > > After upgrading to 6.3, I've encountered a problem while using gif(4): > > Pre-Cond: fresh install of OpenBSD 6.3 > # cat /etc/hostname.gif0 > tunnel 10.0.0.1 10.0.0.2 > 10.0.0.1 10.0.0.2 > # sh /etc/netstart.sh gif0 > # ping 10.0.0.2 > > > Result & Behavior: the kernel thread softnet starts running on the cpu > (STATE: run) and consumes 100% cpu time spent in system mode. The > memory consumption increases until there is no free memory (mbufpl, > mtagpl seem to grow). Even upon stopping/suspending the ping(1) process, > the sofnet thread continues to consume cpu time. Only destroying the > gif0 interface changes the softnet STATE to sleep (memory seems still > allocated). > > I've discovered this on a Hetzner virtual machine (VPS) System (dmesg > and sysinfo attached) but i also reproduced the behavior on a Thinkpad > X200 (dmesg not attached). > > Has anyone seen something similar or is there something I am missing? I > tried to backtrack (at least a bit) and I can reproduce the behavior at > least up to revision 1.109 (if_gif.c). Not sure if my config is absolute > correct but in anyway, I guess softnet shouldn't consume so much cpu > time in that case. can you try this diff? Index: if_gif.c === RCS file: /cvs/src/sys/net/if_gif.c,v retrieving revision 1.113 diff -u -p -r1.113 if_gif.c --- if_gif.c15 Mar 2018 21:01:18 - 1.113 +++ if_gif.c4 Apr 2018 23:12:02 - @@ -403,6 +403,8 @@ gif_output(struct ifnet *ifp, struct mbu error = ENOBUFS; goto drop; } + memcpy((caddr_t)(mtag + 1), &ifp->if_index, sizeof(ifp->if_index)); + m_tag_prepend(m, mtag); m->m_pkthdr.ph_family = dst->sa_family;