We have experienced similar problems - although not with CAS and it is
next to impossible to track these down. In our case they were related
to Cisco Pix firewall rules. If it is just the connections from the
Blackboard service back to CAS for validation, and it were Yale, I
would smell firewall problem. The one case I remember clearly had to
do with inspecting SQL so is clearly not your problem.
Susan
Adam Rybicki wrote:
Many of us have experienced SSL issues when implementing CAS. I have
just stumbled into an issue that is different. It might not even be an
SSL issue...
This is at a fairly large institution. CAS 3.3 is clustered. Many Web
apps use CAS, but only one, Blackboard Vista appears to experience
EXTREME delays when validating tickets with CAS. In the past,
Blackboard Vista was using the legacy Yale CAS client and did not
experience issues. Now the code was upgraded to use the Jasig CAS
client (tried both 3.1.10 and 3.1.11) and some ticket validation
requests take 10 MINUTES to come back. The delay is not the same. When
the call to /cas/serviceValidate comes back in less than 5 minutes, it
is successful. If it takes longer, it says that ticket is
invalid--service tickets expire by default after 5 minutes. All the
delayed calls eventually return.
It is doubtful that any users actually wait long enough to ever get a
successful session. If they simply try again, they may get lucky and
get in normally.
As far as CAS itself is concerned, it is replying instantaneously. CAS
does not appear to be under heavy load, and neither is Blackboard Vista.
Thread dumps from Blackboard Vista show no deadlocks, but there always
are several threads performing a low-level socket read. This is always
from the Jasig CAS client's call to getInputStream(). Curiously, the
stack traces indicate that Weblogic's and some other proprietary code
handles client-side SSL. Weblogic 9.2 uses "Java HotSpot(TM) Server VM
(1.5.0_13-b05 mixed mode)."
And here is an example of one of the stack traces out of the thread dump:
"[ACTIVE] ExecuteThread: '5' for queue: 'weblogic.kernel.Default
(self-tuning)':NTnyMtdJqdtj51FxbXT6:guest" daemon prio=10 tid=0x026783b0
nid=0x7a1 runnable [0x73f7c000..0x73f7fa70]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at
weblogic.utils.io.ChunkedInputStream.read(ChunkedInputStream.java:151)
at java.io.InputStream.read(InputStream.java:89)
at com.certicom.tls.record.ReadHandler.readFragment(Unknown Source)
at com.certicom.tls.record.ReadHandler.readRecord(Unknown Source)
at com.certicom.tls.record.ReadHandler.read(Unknown Source)
- locked <0xa7e1c890> (a com.certicom.tls.record.ReadHandler)
at com.certicom.tls.record.ReadHandler.read(Unknown Source)
at com.certicom.tls.record.ReadHandler.read(Unknown Source)
at
com.certicom.tls.interfaceimpl.TLSConnectionImpl.closeWriteHandler(Unknown
Source)
at com.certicom.tls.interfaceimpl.TLSConnectionImpl.close(Unknown
Source)
at javax.net.ssl.impl.SSLSocketImpl.close(Unknown Source)
at weblogic.net.http.HttpClient.closeServer(HttpClient.java:489)
at weblogic.net.http.HttpClient.findInCache(HttpClient.java:227)
at weblogic.net.http.HttpsClient.New(HttpsClient.java:469)
at
weblogic.net.http.HttpsURLConnection.connect(HttpsURLConnection.java:220)
at
weblogic.net.http.HttpURLConnection.getInputStream(HttpURLConnection.java:342)
- locked <0xa284beb0> (a weblogic.net.http.SOAPHttpsURLConnection)
at
weblogic.net.http.SOAPHttpsURLConnection.getInputStream(SOAPHttpsURLConnection.java:32)
at
org.jasig.cas.client.util.CommonUtils.getResponseFromServer(CommonUtils.java:306)
at
org.jasig.cas.client.util.CommonUtils.getResponseFromServer(CommonUtils.java:285)
at
org.jasig.cas.client.validation.AbstractCasProtocolUrlBasedTicketValidator.retrieveResponseFromServer(AbstractCasProtocolUrlBasedTicketValidator.java:32)
at
org.jasig.cas.client.validation.AbstractUrlBasedTicketValidator.validate(AbstractUrlBasedTicketValidator.java:187)
at
edu.tamu.cas.client.blackboard.CASAuthenticator.validate(CASAuthenticator.java:216)
at
edu.tamu.cas.client.blackboard.TAMUCASModule.login(TAMUCASModule.java:226)
at
com.webct.platform.coreservice.security.authentication.module.ProxyAuthenticationModule.login(ProxyAuthenticationModule.java:135)
at sun.reflect.GeneratedMethodAccessor229.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at javax.security.auth.login.LoginContext.invoke(LoginContext.java:769)
at
javax.security.auth.login.LoginContext.access$000(LoginContext.java:186)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:683)
at java.security.AccessController.doPrivileged(Native Method)
at
javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
at javax.security.auth.login.LoginContext.login(LoginContext.java:579)
at
com.webct.platform.coreservice.security.authentication.common.VistaAgent.authenticate(VistaAgent.java:118)
at
com.webct.platform.coreservice.security.authentication.common.VistaAuthenticationManager.getAgent(VistaAuthenticationManager.java:95)
at
com.webct.platform.coreservice.security.authentication.common.AuthenticationHandler.performSSOAuthentication(AuthenticationHandler.java:271)
at
com.webct.platform.coreservice.security.authentication.common.AuthenticationHandler.handleRequest(AuthenticationHandler.java:109)
at
com.webct.platform.coreservice.action.RequestProcessor.process(RequestProcessor.java:171)
at
org.apache.struts.action.ActionServlet.process(ActionServlet.java:1164)
at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:397)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:743)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
at
weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:225)
at
weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:127)
at
weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:283)
at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:26)
at
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:42)
at
com.webct.platform.coreservice.action.CacheControlFilter.doFilter(CacheControlFilter.java:90)
at
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:42)
at
com.webct.platform.coreservice.action.RequestCleanupFilter.doFilter(RequestCleanupFilter.java:160)
at
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:42)
at
weblogic.servlet.internal.RequestDispatcherImpl.invokeServlet(RequestDispatcherImpl.java:501)
at
weblogic.servlet.internal.RequestDispatcherImpl.forward(RequestDispatcherImpl.java:245)
at
com.webct.platform.coreservice.action.UrlRewritingServlet.service(UrlRewritingServlet.java:76)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
at
weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:225)
at
weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:127)
at
weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:283)
at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:26)
at
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:42)
at
com.webct.platform.coreservice.action.CacheControlFilter.doFilter(CacheControlFilter.java:90)
at
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:42)
at
com.webct.platform.coreservice.action.RequestCleanupFilter.doFilter(RequestCleanupFilter.java:160)
at
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:42)
at
weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3212)
at
weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at
weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121)
at
weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:1983)
at
weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:1890)
at
weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1344)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:181)
This is using Jasig CAS client 3.1.11, but the line numbers are affected
by some code modifications added by me in a futile attempt at debugging
this.
I searched list archives and came back with nothing. There were some
previously documented issues with keep-alive and SSL. The legacy Yale
CAS client even added a "Connection: close" HTTP header in an apparent
attempt to sidestep keep-alive issues. I tried that in the Jasig CAS
client with no luck.
Has anyone experienced something similar? Sadly, this issue does not
occur in test or dev environments, so the actual users are affected. :-(
Thanks,
Adam
--
Susan Bramhall ([email protected])
Senior Developer, Infrastructure Systems and Architecture
Yale University Information Technology Services (ITS)
25 Science Park, 150 Munson St, New Haven, CT 06520
Phone: 203 432 6697
--
You are currently subscribed to [email protected] as: [email protected]
To unsubscribe, change settings or access archives, see http://www.ja-sig.org/wiki/display/JSG/cas-dev
|
