Thanks Marv and Carl,
I am already getting better logging thanks to Carl’s hint.
> Do you see validation failures on this order of magnitude on a regular basis,
> or did you just have a peak on that day?
July 14th was a peak day. But we do get them on regular basis. On one CAS
server we had 4396 validate failures so far this month. On the 14th we had 3352.
> what evidence do you have that validation is failing?
I can spot ticket validation errors in serveral places. On the client side we
see:
####<Jul 20, 2015 7:02:14 PM MDT> <Error> <HTTP> <done.acs.utah.edu> <PIA5>
<[ACTIVE
] ExecuteThread: '10' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS
Kern
el>> <> <> <1437440534106> <BEA-101017>
<[ServletContext@1281965122[app:peoplesoft m
odule:/ path: spec-version:2.5]] Root cause of ServletException.
org.jasig.cas.client.validation.TicketValidationException: CAS Server could not
validate ticket.
at
org.jasig.cas.client.validation.Cas10TicketValidator.parseResponseFromServer(Cas10TicketValidator.java:31)
On the CAS side, in the Tomcat access log we see this:
[14/Jul/2015:11:18:08 -0600] "GET
/cas/validate?ticket=ST-1270230-GeycJ34wc72XkCcXpe1N-casp&service=https%3A%2F%2Futah.equella.ecollege.com%2Flogon.do
HTTP/1.1" 200 4
Where the “200 4” is returning the string “no”
In the audit log file we see: ACTION: SERVICE_TICKET_VALIDATE_FAILED
Here is a better link (PDF) of our architecture:
https://www.lucidchart.com/publicSegments/view/55ae4e2c-328c-4cbd-aa74-36960a00c2b0/image.pdf
Thanks for all the hints.
Bryan Wooten
Tel: (801)585-9323
Email: [email protected]<mailto:[email protected]>
[Identity & Access Management_combined centered]
From: Marvin Addison [mailto:[email protected]]
Sent: Tuesday, July 21, 2015 7:24 AM
To: [email protected]
Subject: Re: [cas-dev] Diagnosing Service Ticket validation errors
Sorry if I am spamming this list but I am desperate.
Yeah, this is a support issue, but we'll cut you some slack ;)
On July 14th we got over 2000 of these errors out of about 30k successful
logins. This led to (thanks ITIL
) awareness up to the VP level. I am under the gun to find a “solution” before
the start of school August 24th.
I think a ~7% ticket validation failure rate is something of legitimate
concern. Do you see validation failures on this order of magnitude on a regular
basis, or did you just have a peak on that day?
I have turned up log level to debug on the CAS servers. I see successful
validations in the logs, but not unsuccessful validations.
Ticket validation failures are indeed logged, both in audit and in the
validator components. Here are some randomly-chosen audit events from our log
for today:
2015-07-21T09:06:02.252|ST-567525-EK0BvF9xYKDqDSHQUCTV-cas2|audit:unknown|SERVICE_TICKET_VALIDATE_FAILED|198.82.164.189
2015-07-21T09:01:33.503|ST-567442-71agV1hMMND2CC4ntePf-cas2|audit:unknown|SERVICE_TICKET_VALIDATE_FAILED|198.82.164.171
2015-07-21T09:00:59.650|null|audit:unknown|SERVICE_TICKET_VALIDATE_FAILED|128.173.56.37
2015-07-21T08:55:42.543|ST-71780-WlfObEPXfSOfnYdWZlzp-cas1|audit:unknown|SERVICE_TICKET_VALIDATE_FAILED|198.82.169.7
2015-07-21T08:51:00.075|AAHnrNAKBfKATlQQH7UKhnKXNdebx13zB0yXtKeDauD1CWNJ1o30W0QV/|audit:unknown|SERVICE_TICKET_VALIDATE_FAILED|198.82.162.156
Now if I understand how CAS works, there can only be 3 reasons an ST won’t
validate: it is being reused, it has timed out or it does not exist / is
corrupted.
Correct. So if you don't have any record of ticket validation failure, what
evidence do you have that validation is failing?
Can someone point me to the method(s) that does the validation?
Several places within the following method you could add debug logging, but you
can see there's already quite a bit:
https://github.com/Jasig/cas/blob/3.5.x/cas-server-core/src/main/java/org/jasig/cas/CentralAuthenticationServiceImpl.java#L338
Here is a diagram of our infrastructure:
https://www.lucidchart.com/invitations/accept/da009b9d-e55f-4f95-9301-e6bd23d508ab
I'll take a closer look at the diagram once I get some more information on how
you're identifying ticket validation failures. You should be getting logging on
the CAS server, and the fact that you are apparently not getting that suggests
a client problem.
M
--
You are currently subscribed to
[email protected]<mailto:[email protected]> as:
[email protected]<mailto:[email protected]>
To unsubscribe, change settings or access archives, see
http://www.ja-sig.org/wiki/display/JSG/cas-dev
--
You are currently subscribed to [email protected] as:
[email protected]
To unsubscribe, change settings or access archives, see
http://www.ja-sig.org/wiki/display/JSG/cas-dev
