Our Aruba SE alerted us but we are running 6.6.5. Out servers already had the fixed version anyway.
For RADIUS monitoring we use Nagios and monitor twice. One services uses an Active Directory service account, and a second one uses a ClearPass local user account. Aruba recommends this to assist is problem isolation if there is a failure. Bruce Osborne Senior Network Engineer Network Operations - Wireless (434) 592-4229 LIBERTY UNIVERSITY Training Champions for Christ since 1971 From: Ferguson, Michael [mailto:[email protected]] Sent: Wednesday, October 11, 2017 1:01 PM Subject: Re: Clearpass Bug - Posture and Profile Data update Unfortunately, we were hit by the same bug as Chad and possibly a few others on the list. It looks like the problem affects Clearpass customers running 6.6.7+. We struggled to find a fix early this morning and finally got services up around 7:15 am pacific time once we identified the issue. But until we were noticed the problem and resolved it, we were down for wireless access across campus for 6 hours due to this Clearpass bug—the issue started at 1 am for us. This brings up an obvious need on our part to check our Clearpass servers from a 3rd-party tool for authentication successes and failures. I think we’ll have to use a Nagios plugin (or something like it) for radius authentication checks, which I didn’t expect we would need to do. As for monitoring other processes on individual Clearpass servers, I don’t have a ready answer on that one. However, this does bring up a desire on my part related to vendor participation on the list. I know we have some HPE/Aruba employees that participate on the list and I think the Wireless-LAN group would be a perfect vehicle for them to disseminate information to customers that could be affected by known issues, particularly ones that could impact services to your campus. When we had the issue this morning, one of the places I looked was the Wireless-LAN discussions to see if anyone was affected by problems with Clearpass. I didn’t see any (until Chad posted later) and so we thought our issue was more isolated. We wasted 20 minutes of valuable MTTR time collecting Server Logs when all we needed to do was start the “Policy server” service. However, if I had seen a post from HPE/Aruba to the Wireless-LAN list about a possible problem affecting many customers, we could’ve started working on the real issue earlier. Putting in a TAC case related to a critical 1 issue is something we generally wait to do if we can’t find a quick fix on our side. -- Mike Ferguson Chapman University Network Manager 714-744-7873 [email protected]<mailto:[email protected]> From: The EDUCAUSE Wireless Issues Constituent Group Listserv [mailto:[email protected]] On Behalf Of Amel Caldwell Sent: Wednesday, October 11, 2017 9:05 AM To: [email protected]<mailto:[email protected]> Subject: Re: [WIRELESS-LAN] Clearpass Bug - Posture and Profile Data update Fortunately for us, we are still on 6.6.5 and we were not affected by this. This did make me think about how fragile the operational state of the ClearPass cluster can be. Looking through my event logs, I see the AV/AS updates happening 20 plus time a day and they hit all of our servers simultaneously. So, I am curious how others deal with this. Do you monitor process status on each of your individual servers? Do you have automated mechanisms to restart stopped processes and notify engineers? If so, what methods do you use? Amel Caldwell University of Washington UW-IT Wi-Fi Network Engineer Wi-Fi Service Manager [email protected]<mailto:[email protected]> 206-543-2915 University of Washington has open positions for Wi-Fi Network Engineers on our Network Design and Architecture team. https://uwhires.admin.washington.edu/ENG/candidates/default.cfm?szCategory=jobprofile&szOrderID=147382&szCandidateID=0&szSearchWords=&szReturnToSearch=1<https://urldefense.proofpoint.com/v2/url?u=https-3A__uwhires.admin.washington.edu_ENG_candidates_default.cfm-3FszCategory-3Djobprofile-26szOrderID-3D147382-26szCandidateID-3D0-26szSearchWords-3D-26szReturnToSearch-3D1&d=DwMGaQ&c=TwQYWVcq0sGbkW5mKeqBpQ&r=ueO6Ax6pfjgBKq_ZIVlkKRBwTNvuR0XFPfu97IwKx3Q&m=zO-SbvfxeY5NctRcVM8EKtWg1-FsLscwFXIIrLOQ00I&s=jHfvMdWAXR2WtqLOmULM4GG3ejjjF3yMOJsxZ6FqN2Q&e=> https://uwhires.admin.washington.edu/ENG/candidates/default.cfm?szCategory=jobprofile&szOrderID=147172&szCandidateID=0&szSearchWords=&szReturnToSearch=1<https://urldefense.proofpoint.com/v2/url?u=https-3A__uwhires.admin.washington.edu_ENG_candidates_default.cfm-3FszCategory-3Djobprofile-26szOrderID-3D147172-26szCandidateID-3D0-26szSearchWords-3D-26szReturnToSearch-3D1&d=DwMGaQ&c=TwQYWVcq0sGbkW5mKeqBpQ&r=ueO6Ax6pfjgBKq_ZIVlkKRBwTNvuR0XFPfu97IwKx3Q&m=zO-SbvfxeY5NctRcVM8EKtWg1-FsLscwFXIIrLOQ00I&s=b_J7ALuJkLdjXsyqKhT4v9Fn1Gwopymkxx1u-dx8aDQ&e=> From: The EDUCAUSE Wireless Issues Constituent Group Listserv <[email protected]<mailto:[email protected]>> on behalf of Chad Burnham <[email protected]<mailto:[email protected]>> Reply-To: The EDUCAUSE Wireless Issues Constituent Group Listserv <[email protected]<mailto:[email protected]>> Date: Wednesday, October 11, 2017 at 8:43 AM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: [WIRELESS-LAN] Clearpass Bug - Posture and Profile Data update HI fellow Clearpass users: This one bit us this morning. Not a great way to come into work today. The Posture and Profile Data update version 1.48743 which was released today had caused the Policy Service to crash causing authentication issues. A defect RM42553 has been created for this issue. The Dev Team has released an update 1.48751 which has resolved the issue. Please ensure that the update 1.48751 is installed and the Policy Service is running on all the servers in the cluster, by following the below stated steps. · To install AV/AS Update version 1.48751, Please navigate to ClearPass Policy Manager GUI à Administration à Agents and Software Updates àSoftware Updates page à Click on 'Check Status Now". · Please navigate to ClearPass Policy Manager GUI à Administration àServer Manager à Server Configuration à Click on the name of the serverà Services Control à Check for the status of the Policy server. · If the status is Stopped, please click on the Start button next to it, to start the service. The ClearPass Dev Team will provide an RCA for this issue shortly. Chad Director of Network Services Information Technology University of Denver 2100 S. High St. #106 Denver, CO 80208 SIP URI = [email protected]<mailto:[email protected]> Desk Phone: 303-871-4441 Mobile Phone: 303-520-5657 https://du.webex.com/join/cburnham<https://urldefense.proofpoint.com/v2/url?u=https-3A__du.webex.com_join_cburnham&d=DwMGaQ&c=TwQYWVcq0sGbkW5mKeqBpQ&r=ueO6Ax6pfjgBKq_ZIVlkKRBwTNvuR0XFPfu97IwKx3Q&m=zO-SbvfxeY5NctRcVM8EKtWg1-FsLscwFXIIrLOQ00I&s=pqWHpMiYtbtcgd-9-xlY8ScxSY9WLDsrHpWAGDAoMKM&e=> https://udenver.zoom.us/my/cburnham<https://urldefense.proofpoint.com/v2/url?u=https-3A__udenver.zoom.us_my_cburnham&d=DwMGaQ&c=TwQYWVcq0sGbkW5mKeqBpQ&r=ueO6Ax6pfjgBKq_ZIVlkKRBwTNvuR0XFPfu97IwKx3Q&m=zO-SbvfxeY5NctRcVM8EKtWg1-FsLscwFXIIrLOQ00I&s=i50lMkehuMSw4SB7k2Jvn6-MWbeDIWhClX0DWts70qs&e=> ********** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at http://www.educause.edu/discuss<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.educause.edu_discuss&d=DwMGaQ&c=TwQYWVcq0sGbkW5mKeqBpQ&r=ueO6Ax6pfjgBKq_ZIVlkKRBwTNvuR0XFPfu97IwKx3Q&m=zO-SbvfxeY5NctRcVM8EKtWg1-FsLscwFXIIrLOQ00I&s=pJbTiQaPEtSMij41gd-nxFwCrz5Qrh9Kc_aw6cTqDZo&e=>. ********** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at http://www.educause.edu/discuss<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.educause.edu_discuss&d=DwMGaQ&c=TwQYWVcq0sGbkW5mKeqBpQ&r=ueO6Ax6pfjgBKq_ZIVlkKRBwTNvuR0XFPfu97IwKx3Q&m=zO-SbvfxeY5NctRcVM8EKtWg1-FsLscwFXIIrLOQ00I&s=pJbTiQaPEtSMij41gd-nxFwCrz5Qrh9Kc_aw6cTqDZo&e=>. ********** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at http://www.educause.edu/discuss. ********** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at http://www.educause.edu/discuss.
