Hi Chad, We have an 8 node cluster with roughly 3,500 to 4,000 users on each node. In this case “users” really means IP addresses the way they get counted. So in a dual IPv4/IPv6 stack client environment, a single user may show up as 3 “users” with the IPv4 address, IPv6 link local, and globally routable IPv6 address.
Eric Kenny Network Architect | Technology Partner Services Harvard University Information Technology On Sep 3, 2021, at 12:17 PM, Street, Chad A <[email protected]<mailto:[email protected]>> wrote: For anyone who has applied the suggested fixes and are still having problems -- what are your client loads per controller? I am also very curious on any feedback for anyone who has applied the fixes and has also successfully enabled Airwave ( not central ) without issues reoccurring. Chad [email protected]<mailto:[email protected]> ________________________________ From: The EDUCAUSE Wireless Issues Community Group Listserv <[email protected]<mailto:[email protected]>> on behalf of Patrick McEvilly <[email protected]<mailto:[email protected]>> Sent: Friday, September 3, 2021 12:03 PM To: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> Subject: Re: [WIRELESS-LAN] [External] Re: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13) A quick update on where we are at. We had many of the same issues being reported and did the same workarounds. However, we ran into a few further issues that others might want to be aware of. We had one AP group that did not have our AP crash dump server configured and was using the "default" which is the controller itself. ~300 APs in that AP group had core dumped and were trying to write to the controller. There is some mechanism that checks for available space on the controller before dumping. This was jamming up the same queue that is used for other critical processes such as STM. This caused a huge problem for us. We changed this one AP group to point to our dump server and saw immediate relief. All APs came back online and clients reconnected. Two hours later we dropped to ~100 APs connected and only 17 clients (yes 17, not 17,000) on our wireless network. We had a cluster event that triggered the controllers to send an invalid IPv4 address to all APs which in turn caused every AP to go into a core dump loop (that was not fun). The only recovery option available was to reboot all controllers. It is believed this is related to 8.7 now supporting IPv6 in the clustering configuration. Our controllers are dual stack but our APs are not. It is possible if the APs were dual stacked they would have used IPv6 and continued to operate over IPv6. Per Aruba removing IPv6 from the controllers is also not an option, we could still experience this problem. This one was a real burn and we continue to be susceptible to this condition. We are anticipating an emergency code release from Aruba in the next few days. Patrick On 9/3/21, 11:20 AM, "The EDUCAUSE Wireless Issues Community Group Listserv on behalf of Jerry Bucklaew" <[email protected]<mailto:[email protected]> on behalf of [email protected]<mailto:[email protected]>> wrote: Scott, I have heard it worked in several places, but we are like you. Put in all the recommended changes last night and it is still happening. -----Original Message----- From: The EDUCAUSE Wireless Issues Community Group Listserv <[email protected]<mailto:[email protected]>> On Behalf Of Scott Swartzentruber Sent: Friday, September 3, 2021 10:41 AM To: [email protected]<mailto:[email protected]> Subject: Re: [WIRELESS-LAN] [External] Re: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13) Has anyone had success in resolving the client drops we have been discussing? We had Aruba on the phone very early this morning - put the recommended ACL in place and did a a rolling restart of all 3 of our controllers. We only had to wait to our first class change to see a huge drop of clients and another at the next class change. Seems clear that the change and the reboot made no difference. back on the phone with Aruba now to see what else they come up with... I don't like this ride anymore - can I get off now? ************************************ Western Carolina University Scott Swartzentruber Director, Networking and Communications Forsyth Bldg, Room B-16 828.227.3212 [email protected]<mailto:[email protected]> ************************************ ********** Replies to EDUCAUSE Community Group emails are sent to the entire community list. If you want to reply only to the person who sent the message, copy and paste their email address and forward the email reply. Additional participation and subscription information can be found at https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.educause.edu%2Fcommunity&data=04%7C01%7Ccstree2%40EMORY.EDU%7C7010b9354b0144becdd908d96ef458f7%7Ce004fb9cb0a4424fbcd0322606d5df38%7C0%7C0%7C637662818017617110%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Py3VDUVjpo1WIQ0y8v3PmIk%2BhoQi4TbaMfP0bDNopCw%3D&reserved=0<https://urldefense.proofpoint.com/v2/url?u=https-3A__nam11.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Fwww.educause.edu-252Fcommunity-26amp-3Bdata-3D04-257C01-257Ccstree2-2540EMORY.EDU-257C7010b9354b0144becdd908d96ef458f7-257Ce004fb9cb0a4424fbcd0322606d5df38-257C0-257C0-257C637662818017617110-257CUnknown-257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0-253D-257C1000-26amp-3Bsdata-3DPy3VDUVjpo1WIQ0y8v3PmIk-252BhoQi4TbaMfP0bDNopCw-253D-26amp-3Breserved-3D0&d=DwMGaQ&c=WO-RGvefibhHBZq3fL85hQ&r=_o-g4xpq_iR_fObGXF1reRWocVpqLeK4U6OqQh1XxBs&m=6CUyAeXg1YiJXeq26Qx5sn758lX54qTJLU-__40U184&s=aS1PBPEqdxH89Oc3dCi0IA1vKndQVuiBSLsT_Rmks4g&e=> ********** Replies to EDUCAUSE Community Group emails are sent to the entire community list. If you want to reply only to the person who sent the message, copy and paste their email address and forward the email reply. Additional participation and subscription information can be found at https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.educause.edu%2Fcommunity&data=04%7C01%7Ccstree2%40EMORY.EDU%7C7010b9354b0144becdd908d96ef458f7%7Ce004fb9cb0a4424fbcd0322606d5df38%7C0%7C0%7C637662818017617110%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Py3VDUVjpo1WIQ0y8v3PmIk%2BhoQi4TbaMfP0bDNopCw%3D&reserved=0<https://urldefense.proofpoint.com/v2/url?u=https-3A__nam11.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Fwww.educause.edu-252Fcommunity-26amp-3Bdata-3D04-257C01-257Ccstree2-2540EMORY.EDU-257C7010b9354b0144becdd908d96ef458f7-257Ce004fb9cb0a4424fbcd0322606d5df38-257C0-257C0-257C637662818017617110-257CUnknown-257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0-253D-257C1000-26amp-3Bsdata-3DPy3VDUVjpo1WIQ0y8v3PmIk-252BhoQi4TbaMfP0bDNopCw-253D-26amp-3Breserved-3D0&d=DwMGaQ&c=WO-RGvefibhHBZq3fL85hQ&r=_o-g4xpq_iR_fObGXF1reRWocVpqLeK4U6OqQh1XxBs&m=6CUyAeXg1YiJXeq26Qx5sn758lX54qTJLU-__40U184&s=aS1PBPEqdxH89Oc3dCi0IA1vKndQVuiBSLsT_Rmks4g&e=> ********** Replies to EDUCAUSE Community Group emails are sent to the entire community list. If you want to reply only to the person who sent the message, copy and paste their email address and forward the email reply. Additional participation and subscription information can be found at https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.educause.edu%2Fcommunity&data=04%7C01%7Ccstree2%40EMORY.EDU%7C7010b9354b0144becdd908d96ef458f7%7Ce004fb9cb0a4424fbcd0322606d5df38%7C0%7C0%7C637662818017617110%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Py3VDUVjpo1WIQ0y8v3PmIk%2BhoQi4TbaMfP0bDNopCw%3D&reserved=0<https://urldefense.proofpoint.com/v2/url?u=https-3A__nam11.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Fwww.educause.edu-252Fcommunity-26amp-3Bdata-3D04-257C01-257Ccstree2-2540EMORY.EDU-257C7010b9354b0144becdd908d96ef458f7-257Ce004fb9cb0a4424fbcd0322606d5df38-257C0-257C0-257C637662818017617110-257CUnknown-257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0-253D-257C1000-26amp-3Bsdata-3DPy3VDUVjpo1WIQ0y8v3PmIk-252BhoQi4TbaMfP0bDNopCw-253D-26amp-3Breserved-3D0&d=DwMGaQ&c=WO-RGvefibhHBZq3fL85hQ&r=_o-g4xpq_iR_fObGXF1reRWocVpqLeK4U6OqQh1XxBs&m=6CUyAeXg1YiJXeq26Qx5sn758lX54qTJLU-__40U184&s=aS1PBPEqdxH89Oc3dCi0IA1vKndQVuiBSLsT_Rmks4g&e=> ********** Replies to EDUCAUSE Community Group emails are sent to the entire community list. If you want to reply only to the person who sent the message, copy and paste their email address and forward the email reply. Additional participation and subscription information can be found at https://www.educause.edu/community<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.educause.edu_community&d=DwMGaQ&c=WO-RGvefibhHBZq3fL85hQ&r=_o-g4xpq_iR_fObGXF1reRWocVpqLeK4U6OqQh1XxBs&m=6CUyAeXg1YiJXeq26Qx5sn758lX54qTJLU-__40U184&s=40_vi6RBsOnaPqJhPB8A-ydWhI0ucdCC96XR0W8I9-I&e=> ********** Replies to EDUCAUSE Community Group emails are sent to the entire community list. If you want to reply only to the person who sent the message, copy and paste their email address and forward the email reply. Additional participation and subscription information can be found at https://www.educause.edu/community
