We have had poor wifi at our university since school started August 10. We have less students than we did last semester, and less students in classes because of social distancing. I am not the network person. However I know I have had good luck at finding answers from other people, so I thought I would share this with you all to see if you have any input. Most of this means nothing to me. Would be happy for any suggestions you have! Below are the two emails that the IT department just sent - one to me and one to the whole campus. Thanks again for any input. ~~~~~~~~~~~~~~~~~~~~~~~~~~ Let me just say in non-technical terms, I have heard that there is a point that the traffic through the access points just stops, and my understanding is that the Ubiquiti APs get super hot and some have failed. Some will work after cooling down for several minutes. The HP and Ubiquitis are reacting the same, but the Ubiquitis are worse. We did just switch from Moodle to Canvas, and some classes are being taught via Zoom.
~~~~~~~~~~~~~~~~~~~~~~~~~~ Information Systems would like to give an update on what we have found, so far, in our work to solve the WiFi problems that have been experienced this semester. Our working theory is that over the summer updates to the networking for computers and mobile devices have changed to include new features. Occasionally, the new features cause problems with our local infrastructure. Right now our wireless network is overloaded with a lot of unnecessary traffic. On a small network with a few devices such as a home, this traffic would not be a problem and is very useful for interacting with printers, cameras, or other devices. On a large network with 1800 devices just on the student network, it can be a big problem. We are trying to resolve how to control this traffic. This does not have anything to do with our Internet connection speed, which is doing very well. It is mainly centered around activity happening on the student network, which of course is our largest network. We have been continually testing and making changes to our network. Some of you may have seen us monitoring classes or even asking people in a class to turn off or on certain devices. While there is not much activity for us to monitor on Tuesdays and Thursdays, Mondays, Wednesdays, and Fridays are busy times for testing. We made some significant configuration changes and will be testing them tomorrow. Below are some details for the technically curious: Here is what we know: - Very high volume of multicast traffic - Seems to be mostly mDNS protocol - IPv4 and IPv6 are used for transport - Affects WiFi more than wired network - Affects both HP and Ubiquiti devices Here is what we think: - Clients have been updated to use newer protocols - mDNS is used mostly to talk to IoT devices - mDNS is similar to AppleTalk's NBP and is very chatty - Our Ubiquiti APs fail, some may not be useful for production anymore - HP APs that are 802.11ac compatible fail but will recover - It seems that 802.11n units are more resilient or at least they can recover on their own - Problem seems to be localized in (four classrooms) Here is what we are doing: - Upgraded firmware on our switches - Changed WiFiTX protection to "No MAC protection" which excludes 803.11b devices - Turned on Spanning Tree and IGMP helpers to WiFi - Changed DTIM to 3 - Downgraded the firmware on our working Ubiquiti APs - Experimenting with replacing all 802.11ac units with .11n devices - Controlling broadcasts at switch and AP level - Disabled mDNS at switch level for IPv4 on capable switches - Trying to disable mDNS over IPv6 at switch level - Considering requesting all clients stop using IPv6 Have received permission from Nicole to disrupt Nursing classes in LLC. Will do so if action is relevant We have done some experiments with turning off all devices or just phones, but need more data. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Email directly to me from networking guy: Here is an article that explains the type of thing we think may be causing problems on our WiFi: https://framebyframewifi.net/2018/01/15/beware-of-mdns-floods-from-buggy-android-clients/ I have done a number of packet captures and found in problem areas we have a high percentage of mdns traffic (Multicast DNS, basically peer to peer chatting and device discovery. As best we can tell we do not need any mdns on our campus for academic functions. Some IoT things in the dorms on the Student network many need this, but not much else. Please note that this is not a problem just for Android on our networks however, in fact most of the mdns traffic we are seeing is more Apple based. It is just an IoT problem of this era. We are doing what we can with the equipment we have to limit the problem. Last night we turned on an mdns filter for our core campus switch, but discovered today that it only filters IPV4 traffic and allows mdns IPV6 traffic (and does not have an IPV6 mdns filter). It appears that the IoT devices happily switched to using mostly IPV6. I am researching what we can do to fix this and am implementing what I can. I also have calls in to HP to see if any of their newer gear has IPV6 mdns filters, etc. I expect to have more solutions soon. Debbie Unterseher Instructional Technologist Associate Professor *Union College* 3800 South 48th Street I Lincoln, NE 68506-4386 Office: 402.486.2600 ext. 2303 I www.ucollege.edu ********** Replies to EDUCAUSE Community Group emails are sent to the entire community list. If you want to reply only to the person who sent the message, copy and paste their email address and forward the email reply. Additional participation and subscription information can be found at https://www.educause.edu/community
