A quick update on where we are at. We had many of the same issues being reported and did the same workarounds. However, we ran into a few further issues that others might want to be aware of.
We had one AP group that did not have our AP crash dump server configured and was using the "default" which is the controller itself. ~300 APs in that AP group had core dumped and were trying to write to the controller. There is some mechanism that checks for available space on the controller before dumping. This was jamming up the same queue that is used for other critical processes such as STM. This caused a huge problem for us. We changed this one AP group to point to our dump server and saw immediate relief. All APs came back online and clients reconnected. Two hours later we dropped to ~100 APs connected and only 17 clients (yes 17, not 17,000) on our wireless network. We had a cluster event that triggered the controllers to send an invalid IPv4 address to all APs which in turn caused every AP to go into a core dump loop (that was not fun). The only recovery option available was to reboot all controllers. It is believed this is related to 8.7 now supporting IPv6 in the clustering configuration. Our controllers are dual stack but our APs are not. It is possible if the APs were dual stacked they would have used IPv6 and continued to operate over IPv6. Per Aruba removing IPv6 from the controllers is also not an option, we could still experience this problem. This one was a real burn and we continue to be susceptible to this condition. We are anticipating an emergency code release from Aruba in the next few days. Patrick On 9/3/21, 11:20 AM, "The EDUCAUSE Wireless Issues Community Group Listserv on behalf of Jerry Bucklaew" <[email protected] on behalf of [email protected]> wrote: Scott, I have heard it worked in several places, but we are like you. Put in all the recommended changes last night and it is still happening. -----Original Message----- From: The EDUCAUSE Wireless Issues Community Group Listserv <[email protected]> On Behalf Of Scott Swartzentruber Sent: Friday, September 3, 2021 10:41 AM To: [email protected] Subject: Re: [WIRELESS-LAN] [External] Re: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13) Has anyone had success in resolving the client drops we have been discussing? We had Aruba on the phone very early this morning - put the recommended ACL in place and did a a rolling restart of all 3 of our controllers. We only had to wait to our first class change to see a huge drop of clients and another at the next class change. Seems clear that the change and the reboot made no difference. back on the phone with Aruba now to see what else they come up with... I don't like this ride anymore - can I get off now? ************************************ Western Carolina University Scott Swartzentruber Director, Networking and Communications Forsyth Bldg, Room B-16 828.227.3212 [email protected] ************************************ ********** Replies to EDUCAUSE Community Group emails are sent to the entire community list. If you want to reply only to the person who sent the message, copy and paste their email address and forward the email reply. Additional participation and subscription information can be found at https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.educause.edu%2Fcommunity&data=04%7C01%7Cjpb%40buffalo.edu%7Cf58d662303514aa7e05808d96eea39ec%7C96464a8af8ed40b199e25f6b50a20250%7C0%7C0%7C637662774536265169%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=QcSqYr93it4Sn%2Fc1GaKHK4AGNRz77g7qR8SZ4EbPpKA%3D&reserved=0 ********** Replies to EDUCAUSE Community Group emails are sent to the entire community list. If you want to reply only to the person who sent the message, copy and paste their email address and forward the email reply. Additional participation and subscription information can be found at https://www.educause.edu/community ********** Replies to EDUCAUSE Community Group emails are sent to the entire community list. If you want to reply only to the person who sent the message, copy and paste their email address and forward the email reply. Additional participation and subscription information can be found at https://www.educause.edu/community
