Last week I attempted to put CAS v4.0.2 into production, things were rolling along until I noticed that old TGTs were being looked up from the previous installation (v3.5.2.1) which was causing a wait to occur essentially bogging down my resources then eventually running out of PermGen space. Attached you'll see the stack traces found in my production log files.
Which then I started seeing a lot of authentication failures and TGTs not being crated in the audit trail. Below is the audit trail of when I attempted to log in. 2015-07-24 07:12:23,757 INFO [edu.msu.is.inspektr.audit.support.MsuLoggingAuditTrailManager] - WHO: audit:unknown WHAT: supplied credentials: [jrw16+password] ACTION: AUTHENTICATION_FAILED APPLICATION: CAS WHEN: Fri Jul 24 07:12:23 CDT 2015 CLIENT IP ADDRESS: x.x.x.x SERVER IP ADDRESS: x.x.x.x 2015-07-24 07:12:23,757 INFO [edu.msu.is.inspektr.audit.support.MsuLoggingAuditTrailManager] - WHO: audit:unknown WHAT: 1 errors, 0 successes ACTION: TICKET_GRANTING_TICKET_NOT_CREATED APPLICATION: CAS WHEN: Fri Jul 24 07:12:23 CDT 2015 CLIENT IP ADDRESS: x.x.x.x SERVER IP ADDRESS: x.x.x.x To note: I ran a load test on my development environment I am not seeing these errors occurring. I have just recently added a CAS node to my development instance and will rerun the load test against it and will review the log files. Since that wasn't a success I noticed that v4.0.3 was available, so I slipped that into my development environment. I uncommented out the KryoTranscoder property of my memcached client thinking this was the reason the waits were occurring from the unregistered class. I get an unable to deserialize object error. See development attachment. Now that I'm done explaining the situation (I hope well enough) here are a few questions. 1. When upgrading CAS, do I need to clear memcached of all the ticket data? If so, is there a command line way to do this? 2. Do you suspect that that the PermGen errors are from the waits while trying to retrieve an old ticket from the ticket registry? 3. I have seen that the unable to deserialize object was asked to be submitted as a bug. Is this still an outstanding bug? (to note, I am able to get the memcached stats) 4. Are there repercussions by commenting out the KyroTranscoder property of the memcached client? We are currently running 4 CAS nodes. Thanks, -Jeremy ________________________ Jeremy Wickham Senior Systems Analyst Mississippi State University [email protected]<mailto:[email protected]> -- You are currently subscribed to [email protected] as: [email protected] To unsubscribe, change settings or access archives, see http://www.ja-sig.org/wiki/display/JSG/cas-user
Development-TicketRegistry-Error.log
Description: Development-TicketRegistry-Error.log
Production-TicketRegistry-Errors.log
Description: Production-TicketRegistry-Errors.log
