Re: [WIRELESS-LAN] strange WLC behavior
>> > After a year of pretty much rock solid behavior we’ve had two instances >> > this week where EAP failed for some or all of the users on our WLC 5508 > >In what way? Clients just wouldn't connect. I didn't find anything in the WLC logs that helped me, but probably I just didn't understand what I was seeing. I did see one iPad that made the user accept the cert for our CA, Entrust, but that's about it. > >> > experiencing the problem, but the WebAuth SSID worked fine. The ACS logs >> > showed “EAP session timed out.” The Windows NPS logs didn’t show any >> > authentication failures. > >How many authentications per second? Is it busier than usual? > We're tiny, only 65 APs, currently about 300 users on EAP SSIDs and max 1500 authentications per hour. Let's see, 1500/3600 is about 0.4 ;-) This started sometime overnight, and our peak period is lunchtime. >Could be a case of the WLC reusing RADIUS session IDs which will >totally break stuff and is a know issue under high numbers of >authentications. > >Cisco have gone some way to fix this issue in the latest 8.x, but >as far as I'm concerned their RADIUS client design is overall >still pretty bad. > >> > After a few hours it fixed itself. I tried a 5508 reboot in one of the >> > instances, and it didn’t appear to help. > >So likely behaviour caused by some external factor, such as the >above. But could be anything like eap timers not tuned well, >wireless issues at the edge, etc. Or backend auth being slow. > >Cheers, > >Matthew I'll try going to 8.0.121.0 this weekend since that's easy, and falling back is easy (usually, knock on wood.) Thanks everyone! John ** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at http://www.educause.edu/groups/.
RE: [WIRELESS-LAN] strange WLC behavior
We moved off that as soon as the 8.0.120.x was out. Make sure your AP's can support 8.x code before you migrate to it. 95% of the issues we had on 7.4 and 7.6 went away once we moved to the new software. If you're not running LAG, that will create problems in the 7.x software. S -Original Message- From: The EDUCAUSE Wireless Issues Constituent Group Listserv [mailto:WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU] On Behalf Of Matthew Newton Sent: Thursday, December 03, 2015 10:37 AM To: WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU Subject: Re: [WIRELESS-LAN] strange WLC behavior On Thu, Dec 03, 2015 at 04:17:12PM +, Oliver Elliott wrote: > The 7.6.x range was buggy as hell so I'm not surprised. Get off there > asap! Not as buggy as 7.4.x... we ran 7.6 for a year quite happily. All Cisco software releases are buggy... just depends on whether the bugs affect your particular environment :) > On 3 December 2015 at 16:15, John York wrote: > > > After a year of pretty much rock solid behavior we’ve had two > > instances this week where EAP failed for some or all of the users on > > our WLC 5508 In what way? > > experiencing the problem, but the WebAuth SSID worked fine. The ACS > > logs showed “EAP session timed out.” The Windows NPS logs didn’t > > show any authentication failures. How many authentications per second? Is it busier than usual? Could be a case of the WLC reusing RADIUS session IDs which will totally break stuff and is a know issue under high numbers of authentications. Cisco have gone some way to fix this issue in the latest 8.x, but as far as I'm concerned their RADIUS client design is overall still pretty bad. > > After a few hours it fixed itself. I tried a 5508 reboot in one of > > the instances, and it didn’t appear to help. So likely behaviour caused by some external factor, such as the above. But could be anything like eap timers not tuned well, wireless issues at the edge, etc. Or backend auth being slow. Cheers, Matthew -- Matthew Newton, Ph.D. Systems Specialist, Infrastructure Services, I.T. Services, University of Leicester, Leicester LE1 7RH, United Kingdom For IT help contact helpdesk extn. 2253, ** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at http://www.educause.edu/groups/. ** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at http://www.educause.edu/groups/.
RE: [WIRELESS-LAN] strange WLC behavior
All Cisco software releases are buggy... just depends on whether the bugs affect your particular environment :) Amen to that, and will say "All software is buggy". We're running 8.0.110.11 now for the past year or so, with no ill effects; with WiSM-2 HA clusters. -Original Message- From: The EDUCAUSE Wireless Issues Constituent Group Listserv [mailto:WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU] On Behalf Of Matthew Newton Sent: Thursday, December 03, 2015 10:37 AM To: WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU Subject: Re: [WIRELESS-LAN] strange WLC behavior On Thu, Dec 03, 2015 at 04:17:12PM +, Oliver Elliott wrote: > The 7.6.x range was buggy as hell so I'm not surprised. Get off there asap! Not as buggy as 7.4.x... we ran 7.6 for a year quite happily. All Cisco software releases are buggy... just depends on whether the bugs affect your particular environment :) > On 3 December 2015 at 16:15, John York wrote: > > > After a year of pretty much rock solid behavior we’ve had two > > instances this week where EAP failed for some or all of the users on > > our WLC 5508 In what way? > > experiencing the problem, but the WebAuth SSID worked fine. The ACS > > logs showed “EAP session timed out.” The Windows NPS logs didn’t > > show any authentication failures. How many authentications per second? Is it busier than usual? Could be a case of the WLC reusing RADIUS session IDs which will totally break stuff and is a know issue under high numbers of authentications. Cisco have gone some way to fix this issue in the latest 8.x, but as far as I'm concerned their RADIUS client design is overall still pretty bad. > > After a few hours it fixed itself. I tried a 5508 reboot in one of > > the instances, and it didn’t appear to help. So likely behaviour caused by some external factor, such as the above. But could be anything like eap timers not tuned well, wireless issues at the edge, etc. Or backend auth being slow. Cheers, Matthew -- Matthew Newton, Ph.D. Systems Specialist, Infrastructure Services, I.T. Services, University of Leicester, Leicester LE1 7RH, United Kingdom For IT help contact helpdesk extn. 2253, ** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at http://www.educause.edu/groups/. !DSPAM:911,56606fe0195231016456774! ** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at http://www.educause.edu/groups/.
Re: [WIRELESS-LAN] strange WLC behavior
On Thu, Dec 03, 2015 at 04:17:12PM +, Oliver Elliott wrote: > The 7.6.x range was buggy as hell so I'm not surprised. Get off there asap! Not as buggy as 7.4.x... we ran 7.6 for a year quite happily. All Cisco software releases are buggy... just depends on whether the bugs affect your particular environment :) > On 3 December 2015 at 16:15, John York wrote: > > > After a year of pretty much rock solid behavior we’ve had two instances > > this week where EAP failed for some or all of the users on our WLC 5508 In what way? > > experiencing the problem, but the WebAuth SSID worked fine. The ACS logs > > showed “EAP session timed out.” The Windows NPS logs didn’t show any > > authentication failures. How many authentications per second? Is it busier than usual? Could be a case of the WLC reusing RADIUS session IDs which will totally break stuff and is a know issue under high numbers of authentications. Cisco have gone some way to fix this issue in the latest 8.x, but as far as I'm concerned their RADIUS client design is overall still pretty bad. > > After a few hours it fixed itself. I tried a 5508 reboot in one of the > > instances, and it didn’t appear to help. So likely behaviour caused by some external factor, such as the above. But could be anything like eap timers not tuned well, wireless issues at the edge, etc. Or backend auth being slow. Cheers, Matthew -- Matthew Newton, Ph.D. Systems Specialist, Infrastructure Services, I.T. Services, University of Leicester, Leicester LE1 7RH, United Kingdom For IT help contact helpdesk extn. 2253, ** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at http://www.educause.edu/groups/.
Re: [WIRELESS-LAN] strange WLC behavior
The 7.6.x range was buggy as hell so I'm not surprised. Get off there asap! On 3 December 2015 at 16:15, John York wrote: > After a year of pretty much rock solid behavior we’ve had two instances > this week where EAP failed for some or all of the users on our WLC 5508 > (7.6.130.0). For some users it uses EAP-PEAP-MSChapV2 to a Windows AD > server running NPS. For others it uses EAP-TLS to Cisco ACS. Both were > experiencing the problem, but the WebAuth SSID worked fine. The ACS logs > showed “EAP session timed out.” The Windows NPS logs didn’t show any > authentication failures. > > > > After a few hours it fixed itself. I tried a 5508 reboot in one of the > instances, and it didn’t appear to help. None of the certs involved have > expired and there haven’t been any recent configuration changes. > > I was going to upgrade to one of Cisco’s suggested WLC software versions > over Christmas break—maybe this weekend would be better. > > > > Thanks > > John > ** Participation and subscription information for this EDUCAUSE > Constituent Group discussion list can be found at > http://www.educause.edu/groups/. > > -- Oliver Elliott Senior Network Specialist IT Services University of Bristol e: oliver.elli...@bristol.ac.uk t: 0117 39 (41131) ** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at http://www.educause.edu/groups/.