Good question. Turns out the APs use UTC time, which appears to be correct:
AP#sh clock
*17:29:03.737 UTC Wed Feb 1 2012
-dan
Dan Brisson
Network Engineer
University of Vermont
(Ph) 802.656.8111
[email protected]
On 2/1/2012 12:11 PM, Craig Eyre wrote:
Dan,
What time is your controller showing? From your log messages it looks like
its 2:16pm there? AP's have issues when the time is off between the
controller and the ap's.
Craig Eyre
Network Analyst
IT Services Department
Mount Royal University
4825 Mount Royal Gate SW
Calgary AB T2P 3T5
P. 403.440.5199
E. [email protected]
"The difference between a successful person and others is not a lack of
strength, not a lack of knowledge, but rather in a lack of will." Vincent
T. Lombardi
From: Dan Brisson<[email protected]>
To: [email protected]
Date: 02/01/2012 09:32 AM
Subject: Re: [WIRELESS-LAN] Cisco APs losing CAPWAP session
Sent by: The EDUCAUSE Wireless Issues Constituent Group Listserv
<[email protected]>
This what we see on the AP at the time the AP disjoins:
*Feb 1 14:16:25.174: %DTLS-5-SEND_ALERT: Send FATAL : Close notify
Alert to 10.246.207.214:5246
*Feb 1 14:16:25.227: %WIDS-5-DISABLED: IDS Signature is removed and
disabled.
*Feb 1 14:16:25.227: %CAPWAP-5-CHANGED: CAPWAP changed state to DISCOVERY
*Feb 1 14:16:25.227: %CAPWAP-5-CHANGED: CAPWAP changed state to DISCOVERY
*Feb 1 14:16:25.293: %LINK-5-CHANGED: Interface Dot11Radio0, changed
state to administratively down
*Feb 1 14:16:25.293: %LINK-5-CHANGED: Interface Dot11Radio1, changed
state to administratively down
*Feb 1 14:16:25.299: %LINK-5-CHANGED: Interface Dot11Radio0, changed
state to reset
*Feb 1 14:16:25.309: status of voice_diag_test from WLC is false
*Feb 1 14:16:25.309: %LINK-3-UPDOWN: Interface Dot11Radio1, changed
state to up
*Feb 1 14:16:25.318: %LINK-3-UPDOWN: Interface Dot11Radio0, changed
state to up
-dan
Dan Brisson
Network Engineer
University of Vermont
(Ph) 802.656.8111
[email protected]
On 2/1/2012 10:30 AM, Mike Goebel wrote:
Dan, have you tried logging into the AP itself and checking the logs
by chance?
Mike
On 2/1/2012 10:03 AM, Dan Brisson wrote:
It does seem as though I've grabbed some folks attention. I sure hope it
turns out to not be something simple. :)
I could certainly try moving the APs around...easy enough to do,
although from what we've seen, the pattern of AP drops is so totally
random, hard to say if I'll see anything. At this point though, it's
worth a shot.
All interfaces clean and no QoS in place.
Not sure if this will come through for everyone, but here's an example
of what I see after an AP drops. This is from the controller, on the
General tab for an AP:
Thanks,
-dan
Dan Brisson
Network Engineer
University of Vermont
(Ph) 802.656.8111
[email protected]
On 2/1/2012 9:26 AM, Garry Peirce wrote:
I think you have some of us all getting curious! ;-)
Could you put a historically stable admin AP onto the 5508 and
vice-versa to
see if behaviors change?
Do we assume that all switchports in the path are showing they're
running
clean?
Any QoS config in place on the switches?
-----Original Message-----
From: The EDUCAUSE Wireless Issues Constituent Group Listserv
[mailto:[email protected]] On Behalf Of Dan Brisson
Sent: Wednesday, February 01, 2012 9:09 AM
To:[email protected]
Subject: Re: [WIRELESS-LAN] Cisco APs losing CAPWAP session
Good to know.
The trunks are actually all 10Gig links, or 90% of them are, so
utilization
is most likely not the case, which I'm able to verify from Cacti
graphs.
The APs are connected to 3560Xs PoE switches that then uplink into
either a
3560E-12D or directly into a 4900M where the 5508s are connected.
Certainly
can't rule out physical layer issue somewhere, although it's so wide
spread
across 2 different 5508s that we would need to have multiple issues.
The other interesting thing for us is that the 500 or so APs on our
admin
side that do not lose their CAPWAP session, join to WiSMs, not 5508s.
Thanks,
-dan
Dan Brisson
Network Engineer
University of Vermont
(Ph) 802.656.8111
[email protected]
On 1/31/2012 8:44 PM, Garry Peirce wrote:
We have ~1400 (1240s->3502's) running 7.0.116 and have no such issues.
I would guess at packet loss as well - some things you might look at:
Are the trunks carrying user/AP traffic seem congested when the APs
drop?
Have you verified there are no duplex issues? It may exhibit itself
more as traffic levels rise.
ResHall switching significantly different than on the admin side?
Probably need further topology, version, config info, but as you've a
case open, the TAC will likely ask the same and help find the
culprit(s)
for you.
-----Original Message-----
From: The EDUCAUSE Wireless Issues Constituent Group Listserv
[mailto:[email protected]] On Behalf Of Dan Brisson
Sent: Tuesday, January 31, 2012 8:30 AM
To:[email protected]
Subject: [WIRELESS-LAN] Cisco APs losing CAPWAP session
I'm curious if any Cisco users out there are experiencing or have
experienced what we're seeing on our campus. This past summer we
installed 3502i's in all of our residence halls - approximately 500
total. Ever since the students have moved in, we will get messages
from WCS stating that "AP XYZ" is down and disassociated from the
controller. When I check out the AP, the uptime is fine, but the
"CAPWAP join time" is for like 30 seconds, or however long it took
me to
check.
We've tracked this and it is totally random as to what AP will drop,
which makes troubleshooting this very tough. The log on the AP isn't
helpful.
I'm working with TAC who suggests that keepalives are getting missed.
I'm not sure why that would be the case since we have another
500 or so APs on the admin side that very rarely drop. Adding to
that, when the students left for break, the AP drops stopped. They
came back, and sure enough, the drops start up again.
I will say that the AP always joins back immediately, but for the time
that it does drop A) I'm sure connectivity is affected in that area
and
B) we get an email.
Anyone experiencing this?
Thanks,
-dan
--
Dan Brisson
Network Engineer
University of Vermont
(Ph) 802.656.8111
[email protected]
**********
Participation and subscription information for this EDUCAUSE
Constituent Group discussion list can be found at
http://www.educause.edu/groups/.
**********
Participation and subscription information for this EDUCAUSE
Constituent
Group discussion list can be found athttp://www.educause.edu/groups/.
**********
Participation and subscription information for this EDUCAUSE
Constituent
Group discussion list can be found athttp://www.educause.edu/groups/.
**********
Participation and subscription information for this EDUCAUSE
Constituent Group discussion list can be found
athttp://www.educause.edu/groups/.
********** Participation and subscription information for this EDUCAUSE
Constituent Group discussion list can be found at
http://www.educause.edu/groups/.
**********
Participation and subscription information for this EDUCAUSE Constituent
Group discussion list can be found at http://www.educause.edu/groups/.
**********
Participation and subscription information for this EDUCAUSE Constituent Group
discussion list can be found at http://www.educause.edu/groups/.
**********
Participation and subscription information for this EDUCAUSE Constituent Group
discussion list can be found at http://www.educause.edu/groups/.
