i do not see anything with unusually large packet counts...
----- Original Message ----- From: CBB - Jay Fuller To: [email protected] Sent: Saturday, June 18, 2016 8:16 PM Subject: Re: [AFMUG] after the storm... ok - more details. here is what i know - i'll try not to be too vague. parts of our network are routed. parts are bridged. parts are NATTED (well, a lot of it is natted) The complaints have come from two towers - but the towers are pretty close to our core. Tower 1 - it actually has two backhauls right now because we are/were testing mimosa and haven't finished switching to one backhaul yet. the tower itself and all associated equipment feeds through a ptp600. One customer on an epmp 2.4 radio reports the following : "Every hour or so it will go down for a minute then back up and either ok or slow" , his definition of slow is it won't run a speedtest... lol. He says " Thanks. It started right after storm the other night." So far, that customer is the only report of an issue on that tower. Now - Tower 2 - downstream from Tower 1 - has the following gear in the following configuation. This tower is feeding a large network (sustained 60-80 meg peak times) . It is on a relatively new Netonix switch with only the new mimosa backhaul coming through tower 1. There is nothing else plugged into the switch except for the mimosa gear connecting to tower 2 and the new backhaul from our fiber on tower 1. That is port 2 and port 3 on an rb2011. That connection goes on and terminates on a mikrotik router that routes a bunch of stuff behind it. Few problem reports on the mimosa or anything behind that router. Port 1 is power - powered from a digital loggers device that is connected to a solar panel for backup power. This site is AC fed but can auto-backup to solar if needed. Port 4 is a 2.4 ubnt radio feeding the tower owner - who is also our customer service manager. His home has gotten upwards of 70% sustained packet loss from that radio. (or something). Problem was temporarily resolved by changing subnets but the problem returned a few hours later. Anytime I connect a laptop to his radio (bypassing his internal router, etc) I get an ip address conflcit on any ip I try. [it is not actually an IP address conflcit, the IP won't take on my netbook and when running ipconfig it shows "duplicate" and won't show the ip assigned). I can get it to assign briefly by powercycling the radio. Port 5 - is feeding another tower owner about 15 miles away that feeds a gated community. That tower owner reports no connectivity since the storm but I can see all the way to his equipment (just like i could see to my customer service manager) - assume he also has 70% packet loss. Port 6 and Port 7 - one is a ubnt 2.4 radio and one is a 900 mhz canopy radio. Both seem to work fine. Ubnt 2.4 is our businest rocket probably passing 24 meg or so. Again, off the mimosa feeding this tower. This is not routed but it is on a different subnet (infrastructure is 10.80.3.0/24, subnet on 2.4 gear and 900 gear is 172.25.61.0/24) The customer on port 4 had a public IP bridged (104.152.x.x/29 or something). So did our customer service manager until I tried putting him on a 172.25.61.x address. Port 8 - feeds another tower that is routed behind a mikrotik. Talked to a customer on that network via facebook tonight - he is fine - no drops, no problems, since replacing gear that was ethernet fried on that tower friday morning. I want to blame this tower. I've disabled ports for 15-20 minutes at a time, especially when troubleshooting our customer service manager's connection. I eventually disabled his private radio and put him on the customer 2.4 rocket. No luck there either. The problem seems closest to this tower. Earlier I did get this report from someone downstream from the mimosas but there are like 200 people downstream+ and this is the only report i've heard : "Our internet (Helicon community) Has been up and down for the past few days. It's taken me nearly 15 minutes to load the facebook page and get this message to you. Phone 256-747-2436 with any other questions. It comes and goes in bits and pieces." I really suspected the customer service manager radio - - that is why i disabled it. it might be his home radio. i might try disabling that next.... any other suggestions? This is my current delima... thank you all! ----- Original Message ----- From: CBB - Jay Fuller To: [email protected] Sent: Saturday, June 18, 2016 7:11 PM Subject: [AFMUG] after the storm... several segments of our network have reported intermittent connectivity. in some cases drops "a few times an hour". I observed at one location coming off a 2.4 loco more dropped packets than good packets. instead of replacing that one radio, i thought i might come here for suggestions. i know a long long time ago we'd heard of ubnt radios losing their mind and going on an all out broadcast storm. i think i remember where a firmware release sensed this and disabled the ethernet port (thus bricking he unit, which is better than it causing chaos). something is causing chaos, but it is not everywhere. and of course, it is not getting on other routed segments... any suggestions other than disabling one port on the switch at the affected location at a time?
