Marvin, Thanks for the suggestion about putting the log level on DEBUG (we had it on ERROR so that passwords werent logged). (I was assuming that ERROR would have caught all of the errors, but in the long run my problem wasn't a true "error") I think I might have found the reason for the behavior I was seeing. I will have to do more thorough testing next week to confirm my hunch, but I am pretty confident I know why things were failing.
I tried to access my CASified web application at 1:41 PM; I could log into CAS but the ticket validation was failing. I knew that my ST was granted on CAS server 1 and that it was being validated on CAS server 2. Checking the logs on CAS1 I noticed that the timestamps on the log entries were incorrect; it was 7 minutes behind the actual time. I checked the time stamps of the log entries on CAS2 and noticed that they were 14 minutes behind the actual time! That means that at the point in time I was looking, there was 7 minutes difference between the two CAS servers! The ST was granted on CAS1 at "1:26 PM" and it was validated on CAS2 at "1:34 PM", resulting in a log message of "Ticket expired" (though the real time was merely seconds apart)!! Apparently the IP address of our time server changed in the middle of July and our server administrators did not update it on our CAS servers, causing the clocks to gradually go out of sync. Since then, for maintenance we have done reboots periodically (always at different times, i.e CAS1 then 5 minutes later CAS2, etc.) also. I think that all of this gradually led to the problem we were having; things ran fine until the two clocks got more than 5 minutes different (by default ST expire after 5 minutes), and this would prove why sometimes logins would succeed. So here are the scenarios and the results (in our scenario CAS1 was 7 minutes slower than CAS2): Browser has affinity to CAS1 & web server has affinity to CAS1 -> Login succeeds (same clock that granted is used to validate) Browser has affinity to CAS2 & web server has affinity to CAS2 -> Login succeeds (same clock that granted is used to validate) Browser has affinity to CAS2 & web server has affinity to CAS1 -> Login succeeds (issue time of in the future, so of course ST hasn't expired) Browser has affinity to CAS1 & web server has affinity to CAS2 -> Login FAILS (issue time > 5 minutes in the past, expired ST) I am going to do tests next week to where I change the clock on one of the servers to confirm my suspicion, but I am pretty confident that this is the reason. (All of the pieces fit together and make sense, though what a strange situation!) If my tests happen to not confirm my suspicion and the problem is something else, I guess I will go from there, but if I don't reply again it means that the clock synchronization was the problem the whole time and I really did have CAS and clustering setup correctly! Again, thanks (Scott & Marvin) SO MUCH for all of the help that you have given, I really appreciate it. Let me know if you have any follow up comments or questions. On Fri, Oct 9, 2009 at 1:21 PM, Marvin Addison <[email protected]>wrote: > > If there doesn’t seem to be anything configured incorrectly, what is > going on with these test scenarios? Why does the ticket get inserted into > the database by CAS 1, fails validation on CAS 2, yet is deleted from the > database by CAS 2? > > I'm fairly certain that answering this question will be key to > resolving your problems. The message sent back to the client is a > generic error for ticket validation failure that could have a number > of causes. The CAS server logs, on the other hand, will give detailed > information on why tickets are failing validation. Please find and > share the relevant log excerpts for the ticket validations you > performed in both the server and manual scenarios. It should be > sufficient to search the logs by ticket ID string, e.g. ST-.... > > M > > -- > You are currently subscribed to [email protected] as: > [email protected] > To unsubscribe, change settings or access archives, see > http://www.ja-sig.org/wiki/display/JSG/cas-user > > -- You are currently subscribed to [email protected] as: [email protected] To unsubscribe, change settings or access archives, see http://www.ja-sig.org/wiki/display/JSG/cas-user
