> Date: Sun, 28 Jan 2024 10:44:25 +0100
> From: Marcus Glocker <mar...@nazgul.ch>
>
> On Sun, Jan 28, 2024 at 12:16:20AM +0100, Hrvoje Popovski wrote:
>
> > On 27.1.2024. 21:01, Marcus Glocker wrote:
> > > On Sat, Jan 27, 2024 at 08:01:09AM +0100, Hrvoje Popovski wrote:
> > >
> > >> On 26.1.2024. 21:56, Marcus Glocker wrote:
> > >>> On Fri, Jan 26, 2024 at 11:41:49AM +0100, Hrvoje Popovski wrote:
> > >>>
> > >>>> I've manage to reproduce TSO em problem on anoter setup, unfortunatly
> > >>>> production.
> > >>>>
> > >>>> Setup is very simple
> > >>>>
> > >>>> em0 - carp <- uplink
> > >>>> em1 - pfsync
> > >>>> ix1 - vlans - carp
> > >>> Would it be possible that you also share an "ifconfig -a hwfeatures" of
> > >>> that box? You can mask the IPs if it's too sensitive.
> > >>>
> > >>> I still try to reproduce the issue here, and for now I can't.
> > >>> Maybe in your full ifconfig output I can see some specifics about your
> > >>> configuration, which makes it more likely to reproduce the issue here.
> > >>>
> > >> Hi,
> > >>
> > >> here's ifconfig from second setup where watchdog is triggered much
> > >> faster.
> > >> Originally in this setup uplink is ix0, I've change that to em0 to see
> > >> would the problem be same as in other setup and it is, and that's good
> > >> because this is pfsync setup for students and I can do whatever I want
> > >> with it :)
> > > Thanks.
> > >
> > > But still, I can do whatever I want on my em(4) I210 box, carp(4),
> > > vlan(4), creating a lot of traffic, I can't reproduce the watchdog which
> > > you are seeing :-( I'm not sure if this is something related to your
> > > I350.
> > >
> > > Also, I can't understand why the watchdog still triggers when you disable
> > > TSO by setting net.inet.tcp.tso=0.
> > >
> > > Just to rule out that you're receiving a MAXMCLBYTES (65536) packet,
> > > while EM_TSO_SIZE (65535) is one byte less, can you please apply this
> > > diff to -current and test it? I doubt it will make a difference, but
> > > I'm running a bit out of ideas here.
> >
> >
> > Hi,
> >
> > with this diff I'm still getting em watchdog
> >
> > Jan 28 00:14:12 bcbnfw1 /bsd: em0: watchdog: head 120 tail 185 TDH 185
> > TDT 120
>
> Thanks for testing again.
>
> I think we might have a generic problem with TSO with the current em(4)
> code and some chips. Referring to this recent FreeBSD commit.
>
> e1000: disable TSO on lem(4) and em(4):
> Disable TSO on lem(4) and em(4) until a ring stall can be debugged.
> https://github.com/freebsd/freebsd-src/commit/797e480cba8834e584062092c098e60956d28180
>
> Can you try this diff to specifically disable TSO for I350 please?
>
> We will need to discuss internally which way to go. I see those
> options currently:
>
> - Entirely pull out the TSO diff.
> - Leave the TSO code in but disable TSO for now (what FreeBSD did).
> - Leave the TSO code in but disable TSO only for chips we see issues
> with (this diff).
Frankly, I think it is time to just pull the diff. Between this issue
and the sparc64 unaligned access thing there is just too much breakage
for relatively little gain (since this is only a gigabit Ethernet).
Cheers,
Mark
> Index: if_em.c
> ===================================================================
> RCS file: /cvs/src/sys/dev/pci/if_em.c,v
> diff -u -p -u -p -r1.370 if_em.c
> --- if_em.c 31 Dec 2023 08:42:33 -0000 1.370
> +++ if_em.c 28 Jan 2024 09:30:59 -0000
> @@ -2013,7 +2013,9 @@ em_setup_interface(struct em_softc *sc)
> if (sc->hw.mac_type >= em_82575 && sc->hw.mac_type <= em_i210) {
> ifp->if_capabilities |= IFCAP_CSUM_IPv4;
> ifp->if_capabilities |= IFCAP_CSUM_TCPv6 | IFCAP_CSUM_UDPv6;
> - ifp->if_capabilities |= IFCAP_TSOv4 | IFCAP_TSOv6;
> + /* XXX: Enabling TSO on I350 causes watchdogs */
> + if (sc->hw.mac_type != em_i350)
> + ifp->if_capabilities |= IFCAP_TSOv4 | IFCAP_TSOv6;
> }
>
> /*
>
>
