In this case remember all other WLC's (9 of them) are authenticating successfully against the same RADIUS servers and same LDAP/AD without problem. Only a single WLC stops authenticating and we see the logs with the duplicate RADIUS ID's coming from that WLC which seems to match the Cisco bug ID CSCuo96366.
We have also done dot1x load testing with the Spirent Test center, eapol_test, etc. and we monitor authentications and graph them down to the second. We are certain it is not a back-end ldap or NTLM auth issue. Thanks, Curtis Larsen University of Utah IT/CIS Sr. Network Engineer ________________________________________ From: The EDUCAUSE Wireless Issues Constituent Group Listserv [[email protected]] on behalf of Wang, Yu [[email protected]] Sent: Thursday, September 24, 2015 10:06 AM To: [email protected] Subject: Re: [WIRELESS-LAN] Cisco WLC RADIUS Packet ID Bug FSU uses Aruba wireless controllers with freeradius but we have seen similar issues in the past during peak hours. Through tests we found out that our backend (ldap) had congestions during peak hours and radius servers had to hold on authentication requests forwarded by wireless controllers and wait for ldap servers responses. Because radius cannot get in time response from ldap to reply back to wireless controllers, wireless controllers resend auth requests thinking the previous ones were lost. To remedy this issue,FSU purchased powerful hardware and installed latest ldap service. We also increased number of radius servers and paired up radius and wireless controller. We haven't seen the issue since then. I created a way to test backend capacity from radius server and will share it here later. Since you mentioned "Symptom: Clients are not able to Authenticate at Peak loads when using FreeRadius.", I suspect that you have congested backend as well at peak load time. Not sure what backend you use at Utah you may need to take a look at it, check its load and logs during peak time. Also check that one WLC to see if it has more load than other WLCs (APs serving high concentrated areas) and move some APs to other WLC. You can also increase timeout value on your controllers so they wait a little longer for radius to response back. Increasing number of radius servers would help too but it'll require proper setup between controllers and radius servers. Yu Wang FSU ________________________________________ From: The EDUCAUSE Wireless Issues Constituent Group Listserv [[email protected]] on behalf of Curtis K. Larsen [[email protected]] Sent: Thursday, September 24, 2015 11:30 AM To: [email protected] Subject: [WIRELESS-LAN] Cisco WLC RADIUS Packet ID Bug Hi Guys, I have a TAC case open on this but It looks like once a week or so when the perfect storm arises we are hitting this one for a couple of minutes: CSCuo96366 --- WLC sends Radius packets with same ID without doing Radius ID check CSCuo96366 Description Symptom: Clients are not able to Authenticate at Peak loads when using FreeRadius. Conditions: Using Freed radius (most susceptible), we observe at high auth rate and if Radius server is not responding to all Radius packets in seq order or if the server is slow, WLC when wraps around 0-255 Radius ID's, it does not do a check when posting new packet. So essentially you have 2 packets with same ID being presented to AAA server. --- The funny thing is that 9 of 10 WLC's are working fine against the same servers at the same time - the problem only happens on one WLC. When it occurs we see this in the logs (Notice the same ID number 253 below) servername radiusd[23964]: Discarding conflicting packet from client (IP of WLC) port 32770 - ID: 253 due to recent request 57345605. servername radiusd[23964]: Discarding conflicting packet from client (IP of WLC) port 32770 - ID: 253 due to recent request 57347264 Wondering if other Cisco WLC customers see this since I know a lot of you are using FreeRADIUS, or FreeRADIUS-based authentication servers. If so, let me know of any solutions and/or work-arounds. Thanks, Curtis Larsen University of Utah IT/CIS Sr. Network Engineer ********** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at http://www.educause.edu/groups/. ********** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at http://www.educause.edu/groups/. ********** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at http://www.educause.edu/groups/.
