For the archives...
The issue turned out to be an old PtP radio that was decommissioned,
but not removed from the tower. The ethernet interface of the radio
would start flapping during certain times of the day. That interface
was connected to a router on an interface still configured for OSPF.
Every time the interface flapped, it would trigged a topology change.
The frequency of the topology changes appeared to be causing sub-
second bursts of packet loss that occurred every 30 seconds. We found
it by turning on OSPF debug logging and tallying the number of topology
changes by originator ID. The one at the top of the list was our
culprit.
hth,
-Kristian
On Fri, 2018-02-23 at 03:23 -0500, RickG wrote:
> +1 on the darned surge protectors!
>
> On Thu, Feb 22, 2018 at 3:37 PM, Josh Luthman
> c.com> wrote:
> > Why do you need a loopback test when the capacity graph says the
> > same thing in real time?
> >
> > I would definitely start by doing a ping between radios and see if
> > it's the wireless link versus ethernet/ports/etc. Last time I saw
> > this, it was caused by a surge suppressor making the link go
> > down/up super fast (I would get about 1 packet lost per 5 minutes).
> >
> >
> > Josh Luthman
> > Office: 937-552-2340
> > Direct: 937-552-2343
> > 1100 Wayne St
> > Suite 1337
> > Troy, OH 45373
> >
> > On Thu, Feb 22, 2018 at 3:34 PM, Kristian Hoffmann
> > .com> wrote:
> > > I wish these had a loopback test, because that would be an easy
> > > way to find out. The wireless capacity during the events is a
> > > nice straight line, so there's no indication that it's a wireless
> > > problem, but that doesn't rule it out either.
> > >
> > > I'll run some tests pinging the radio itself at the same time,
> > > and see if there's loss to the radio as well.
> > >
> > > Generally, OSPF is running over these links, but I've disabled
> > > the OSPF instance and have the same results.
> > >
> > > -Kristian
> > >
> > > On Thu, 2018-02-22 at 08:18 -0500, Josh Luthman wrote:
> > > > Definitely through the wireless link? Not including the
> > > > Ethernet?
> > > >
> > > > Josh Luthman
> > > > Office: 937-552-2340
> > > > Direct: 937-552-2343
> > > > 1100 Wayne St
> > > > Suite 1337
> > > > Troy, OH 45373
> > > >
> > > > On Feb 22, 2018 3:19 AM, "Blair Davis" <[email protected]>
> > > > wrote:
> > > > > What routing protocols are you using across those links?
> > > > >
> > > > > --
> > > > >
> > > > >
> > > > > On 2/22/2018 1:54 AM, Kristian Hoffmann wrote:
> > > > > > Hi,
> > > > > >
> > > > > > This is going to sound crazy, but here we go...
> > > > > >
> > > > > > Has anyone seen packet loss across an AF5x link that
> > > > > occurred exactly
> > > > > > every 30 seconds? I have a couple of links, on completely
> > > > > different
> > > > > > towers, that drop packets every 30 seconds. I can observe
> > > > > it using the
> > > > > > MikroTik traffic generator, or a fast ping (size=1500
> > > > > interval=.05).
> > > > > > Even though they do it on a 30 second interval, it's not at
> > > > > the same
> > > > > > time (intervals are not synchronized). They're not bridged
> > > > > together in
> > > > > > any way, and even do it with no customer traffic on them
> > > > > whatsoever.
> > > > > >
> > > > > > I've tried with and without flow control, every wireless
> > > > > setting,
> > > > > > with/without management vlan, firmware 4.0 and 4.0.3, etc.
> > > > > As they're
> > > > > > connected to RB3011's, I even disabled the MikroTik LCD, as
> > > > > apparently
> > > > > > that's a thing. It's a shot in the dark, but... anyone?
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > -Kristian
> > > > > > _______________________________________________
> > > > > > Ubnt_users mailing list
> > > > > > [email protected]
> > > > > > http://lists.wispa.org/mailman/listinfo/ubnt_users
> > > > >
> > > > > --
> > > > > West Michigan Wireless ISP
> > > > > Allegan, Michigan 49010
> > > > > 269-686-8648
> > > > >
> > > > > A Division of:
> > > > > Camp Communication Services, INC
> > > > >
> > > > > _______________________________________________
> > > > > Ubnt_users mailing list
> > > > > [email protected]
> > > > > http://lists.wispa.org/mailman/listinfo/ubnt_users
> > > > >
> > > > _______________________________________________
> > > > Ubnt_users mailing list
> > > > [email protected]
> > > > http://lists.wispa.org/mailman/listinfo/ubnt_users
> > > _______________________________________________
> > > Ubnt_users mailing list
> > > [email protected]
> > > http://lists.wispa.org/mailman/listinfo/ubnt_users
> > >
> >
> > _______________________________________________
> > Ubnt_users mailing list
> > [email protected]
> > http://lists.wispa.org/mailman/listinfo/ubnt_users
> >
>
>
> --
> -RickG
> _______________________________________________
> Ubnt_users mailing list
> [email protected]
> http://lists.wispa.org/mailman/listinfo/ubnt_users
_______________________________________________
Ubnt_users mailing list
[email protected]
http://lists.wispa.org/mailman/listinfo/ubnt_users
