Re: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread Michael Davis
Is your backend (controllers - Radius) all jumbo frame clean?  We've seen issues with large EAP-TLS packets getting fragmented. We also had a specific OS8 release bug affecting AP-515s specifically, but it seems like we're in perpetual bug-chasing mode so I can't recall what version that was.

RE: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread Turner, Ryan H
Its been a while since I look at that. Would be a good path to check Thank you. From: The EDUCAUSE Wireless Issues Community Group Listserv On Behalf Of Michael Davis Sent: Wednesday, September 1, 2021 11:37 AM To: WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU Subject: Re: [WIRELESS-LAN] Anyone else

RE: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread Enfield, Chuck
We're not having any unusual problems now, but we have in the past. Two suggestions I can offer are: * Search your controller syslog for "Authentication server request timeout". This will tell you if the controllers are sending auth requests and not getting replies back. We've had this

Re: [External] Re: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread Norton, Thomas (Network Operations)
We have seen issues with the Lenovo vantage software causing to many ARPs, on top of the IOS 14 issues, that caused the STM process to crash as well. ARP inspection helped mitigate this quite a bit, but would randomly crash one of our controller due a select few clients hashing there. Ryan -

Re: [External] Re: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread Laramie Combs
HEy Chuck - would you mind sharing that arp limiting client filter with me? We are seeing some new traffic patterns where it looks like user devices are just walking their subnets, and arping for everything -Laramie On Wed, Sep 1, 2021 at 11:47 AM Enfield, Chuck wrote: > We’ve seen the CPU

Re: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread Ian Lyons
Thanks Brad! We noticed the same. And do the same thing for general connectivity - turn off randomization. The Crestron's, after the app is loaded and you go to screen share, (and all privacy settings are made) the client just times out and says, "unable to connect". Session table indicate

RE: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread Turner, Ryan H
In our situation, we actually measure the packets over the wire to judge RADIUS response. I know precisely when I get a RADIUS timeout and what the average RTT as well as average response time for MAC and 802.1X authentications. So I believe out environment is clean. With that said, I am

Re: [WIRELESS-LAN] [External] Re: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread Steve Smith
I wouldn't mind seeing the arp limiting client filter as well. Thank you, Steve Steve Smith Network Administrator II Network and Telecommunications Services Aims Community College 970.339.6565 On Wed, Sep 1, 2021 at 9:57 AM Laramie Combs wrote: > HEy Chuck - would you mind sharing that arp

Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread Turner, Ryan H
This is a stab in the dark. With the University mostly shutdown since the Spring of 2020 (=not operating in standard mode and most people work from home), we got campus upgraded from 6.X to 8.X code base. We've also installed many 515 series APs. We are getting a large number of complaints

Re: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread Ian Lyons
Same environment (6.x to 8.x) but no, radius is fine for us. We are on 535's with older 325/303's. (we are AOS 8.6.0.10 which seems -knock on wood -very stable) Our issue is with IOS 14.7 and Crestron Airmedia2's -I suspect the privacy settings are causing us issues. Same Apple hardware ~

Re: [External] [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread Norton, Thomas (Network Operations)
We’re on 8.6.0.11 and not seeing any issues currently, but also running 225/325s in the majority of our class rooms. We just purchased our first round 5xx access points and two of our LPVs are rung 535, 577, and 534s without issue on 8.7.0.4 Do you guys have the HE bit disabled? I know the

RE: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread Floyd, Brad
Ian, iOS 14.0 introduced private MAC addresses. It was broken and devices spoke with both their real MACs and their private MACs. This caused the controllers to blacklist the devices for ARP spoofing. Once the timer expired, the device reconnected again for a while... 14.0 and 14.1 were broken

RE: [WIRELESS-LAN] [External] [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread Turner, Ryan H
So I should say that while I dropped the 515, most of these classrooms have 300 series. So that part isn’t generally related (or consistent to one model type). Ryan From: The EDUCAUSE Wireless Issues Community Group Listserv On Behalf Of Norton, Thomas (Network Operations) Sent: Wednesday,

RE: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread Cody Ensanian
I'm hearing issues of high cpu utilization for STM on the controllers causing issues. Maybe check your controllers and see if you are seeing the high cpu use for STM. Heard earlier today from our SE that Aruba has "identified the issue and is working on a fix." I suggest opening the TAC case so

RE: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread Enfield, Chuck
We've seen the CPU problem, but I don't think it resulted in Auth problems here. It may have and we just missed it because the more severe problems it caused masked them. BTW, in our case reducing the amount of ARP calmed the CPU. We applied a filter (Thank you Colin Joseph.) to limit the

Re: [External] Re: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread Norton, Thomas (Network Operations)
Same here, enabling arp filtering on the firewall helps greatly. T.J. Norton Wireless Network Architect Network Operations Office: (434) 592-6552 [http://www.liberty.edu/media/1616/40themail/wordmark-for-email.jpg] Liberty University | Training Champions for Christ since 1971 On Sep 1,

RE: [WIRELESS-LAN] [External] Re: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread Enfield, Chuck
Here's what we did. We had a pcap that suggested only about 1% of clients would be affected by this filter, but it cut our ARP almost in half. We made the change last spring in our res halls which were almost fully occupied, and we've not traced user complaints back to this yet.

Re: [WIRELESS-LAN] [External] Re: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread Norton, Thomas (Network Operations)
Hey Laramine/Chuck, The ARP issue most likely the Lenovo Vantage software or IOS 14. Another option outside of filtering is to enable prohibit ip spoofing and arp spoofing. T.J. Norton Wireless Network Architect Network Operations (434) 592-6552

RE: [WIRELESS-LAN] [External] Re: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread Enfield, Chuck
Please see my 12:05 response if you missed it. From: The EDUCAUSE Wireless Issues Community Group Listserv On Behalf Of Steve Smith Sent: Wednesday, September 1, 2021 12:00 PM To: WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU Subject: Re: [WIRELESS-LAN] [External] Re: [WIRELESS-LAN] Anyone else seeing

RE: [WIRELESS-LAN] [External] Re: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread McClintic, Thomas
We had disabled "Prohibit ARP Spoofing" at one point to appease the 14+ code. The issue was resolved in a later release and we enabled it again. We are not currently seeing any issues 8.5.0.11, moving to 13 now. Thanks From: The EDUCAUSE Wireless Issues Community Group Listserv On Behalf Of

Re: [WIRELESS-LAN] [External] Re: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread Patrick McEvilly
Just a quick update that might be good information for others to have. We were running 8.7.1.3 since June 10th with no issues until yesterday. We were seeing STM crash across several controllers and caused a significant outage in our wireless service. We did the same things others here were

Re: [WIRELESS-LAN] [External] Re: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread Tolka, Bryan
Ryan, Do you have multicast enabled ?What is the mandatory rate you are using in the classrooms? We just had some issues with this not on Aruba . Bryan Tolka Sent from my iPhone On Sep 1, 2021, at 5:00 PM, Street, Chad A wrote:  Cody and all... We are also seeing STM spikes that are

RE: [WIRELESS-LAN] [External] Re: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread Turner, Ryan H
Glad I brought this up. Is it possible that Cisco environments have evaded this? Seems as though the ARP flooding via iOS 14 would be something that would menace all the manufacturers. From: The EDUCAUSE Wireless Issues Community Group Listserv On Behalf Of Street, Chad A Sent: Wednesday,

RE: [WIRELESS-LAN] [External] Re: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread Cody Ensanian
To all chiming in regarding the Aruba issues - thank you! I love seeing the collaboration and detail sharing. Chad - will be curious to hear if you push the band-aids to production and re-enable airwaves, if this helps your situation. -Cody From: The EDUCAUSE Wireless Issues Community Group

Re: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread Doug Wussler
This happened to us on the first day of classes. It's not your RADIUS. Some problem where, under load, users cannot associate to the AP. The STM process is overcommitted and can't respond appropriately. Aruba advised us to shutdown openflow and the next day the problem was gone. The next

Re: [External] Re: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread Street, Chad A
Cody and all... We are also seeing STM spikes that are impacting associations. We have also disabled all our polling ( Airwave, Orion, etc ) and reduced the client load balancing thresholds so that we have around 4K clients per controller. This seemed to help a great deal. After working

RE: [WIRELESS-LAN] [External] Re: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread Enfield, Chuck
We feel your pain, Patrick! Keep up the good fight. From: The EDUCAUSE Wireless Issues Community Group Listserv On Behalf Of Patrick McEvilly Sent: Wednesday, September 1, 2021 5:25 PM To: WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU Subject: Re: [WIRELESS-LAN] [External] Re: [WIRELESS-LAN] Anyone else

Re: [WIRELESS-LAN] [External] Re: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread Norton, Thomas (Network Operations)
Don't forgot the nasty Lenovo vantage software  T.J. Norton Wireless Network Architect Network Operations (434) 592-6552 [http://www.liberty.edu/media/1616/40themail/wordmark-for-email.jpg] Liberty University | Training Champions for Christ since From:

RE: [WIRELESS-LAN] [External] Re: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread Enfield, Chuck
I'm speculating a bit, but Aruba does a lot of stuff with ARP if features like bcast ARP suppression, convert bacst to unicast, and BC/MC optimization are enabled. I assume Cisco has some similar features, but perhaps not all of them? Or maybe one key feature is causing most of the trouble

Re: [WIRELESS-LAN] [External] Re: [WIRELESS-LAN] Anyone else seeing any issues in the fall with large classrooms and delayed connection times (Aruba 8.5.0.13)

2021-09-01 Thread Patrick McEvilly
I will hold off on providing details for now but when you have to push a code upgrade in the middle of the day on the first day class it’s been a rough day. We hit some major issues related to STM and then other fall out after doing the required code upgrade. We pushed the changes below at