This issue was solved over two months ago, so I apologize for not posting here, as there is a small possibility that someone will run into a similar issue.
The problem turned out to be with WebLogic application server 9.2. Installing the maintenance pack 3 version of WebLogic 9.2 solved the issue. The problem was in some low-level SSL code. That version of the application server does not use standard SSL/TLS support in the JDK. Adam On 7/6/2010 9:47, Susan Bramhall wrote: > We have experienced similar problems - although not with CAS and it is > next to impossible to track these down. In our case they were related > to Cisco Pix firewall rules. If it is just the connections from the > Blackboard service back to CAS for validation, and it were Yale, I > would smell firewall problem. The one case I remember clearly had to > do with inspecting SQL so is clearly not your problem. > Susan > > Adam Rybicki wrote: >> Many of us have experienced SSL issues when implementing CAS. I have >> just stumbled into an issue that is different. It might not even be an >> SSL issue... >> >> This is at a fairly large institution. CAS 3.3 is clustered. Many Web >> apps use CAS, but only one, Blackboard Vista appears to experience >> EXTREME delays when validating tickets with CAS. In the past, >> Blackboard Vista was using the legacy Yale CAS client and did not >> experience issues. Now the code was upgraded to use the Jasig CAS >> client (tried both 3.1.10 and 3.1.11) and some ticket validation >> requests take 10 MINUTES to come back. The delay is not the same. When >> the call to /cas/serviceValidate comes back in less than 5 minutes, it >> is successful. If it takes longer, it says that ticket is >> invalid--service tickets expire by default after 5 minutes. All the >> delayed calls eventually return. >> >> It is doubtful that any users actually wait long enough to ever get a >> successful session. If they simply try again, they may get lucky and >> get in normally. >> >> As far as CAS itself is concerned, it is replying instantaneously. CAS >> does not appear to be under heavy load, and neither is Blackboard Vista. >> >> Thread dumps from Blackboard Vista show no deadlocks, but there always >> are several threads performing a low-level socket read. This is always >> from the Jasig CAS client's call to getInputStream(). Curiously, the >> stack traces indicate that Weblogic's and some other proprietary code >> handles client-side SSL. Weblogic 9.2 uses "Java HotSpot(TM) Server VM >> (1.5.0_13-b05 mixed mode)." >> >> And here is an example of one of the stack traces out of the thread dump: >> >> "[ACTIVE] ExecuteThread: '5' for queue: 'weblogic.kernel.Default >> (self-tuning)':NTnyMtdJqdtj51FxbXT6:guest" daemon prio=10 tid=0x026783b0 >> nid=0x7a1 runnable [0x73f7c000..0x73f7fa70] >> at java.net.SocketInputStream.socketRead0(Native Method) >> at java.net.SocketInputStream.read(SocketInputStream.java:129) >> at >> weblogic.utils.io.ChunkedInputStream.read(ChunkedInputStream.java:151) >> at java.io.InputStream.read(InputStream.java:89) >> at com.certicom.tls.record.ReadHandler.readFragment(Unknown Source) >> at com.certicom.tls.record.ReadHandler.readRecord(Unknown Source) >> at com.certicom.tls.record.ReadHandler.read(Unknown Source) >> - locked<0xa7e1c890> (a com.certicom.tls.record.ReadHandler) >> at com.certicom.tls.record.ReadHandler.read(Unknown Source) >> at com.certicom.tls.record.ReadHandler.read(Unknown Source) >> at >> com.certicom.tls.interfaceimpl.TLSConnectionImpl.closeWriteHandler(Unknown >> Source) >> at com.certicom.tls.interfaceimpl.TLSConnectionImpl.close(Unknown >> Source) >> at javax.net.ssl.impl.SSLSocketImpl.close(Unknown Source) >> at weblogic.net.http.HttpClient.closeServer(HttpClient.java:489) >> at weblogic.net.http.HttpClient.findInCache(HttpClient.java:227) >> at weblogic.net.http.HttpsClient.New(HttpsClient.java:469) >> at >> weblogic.net.http.HttpsURLConnection.connect(HttpsURLConnection.java:220) >> at >> weblogic.net.http.HttpURLConnection.getInputStream(HttpURLConnection.java:342) >> - locked<0xa284beb0> (a weblogic.net.http.SOAPHttpsURLConnection) >> at >> weblogic.net.http.SOAPHttpsURLConnection.getInputStream(SOAPHttpsURLConnection.java:32) >> at >> org.jasig.cas.client.util.CommonUtils.getResponseFromServer(CommonUtils.java:306) >> at >> org.jasig.cas.client.util.CommonUtils.getResponseFromServer(CommonUtils.java:285) >> at >> org.jasig.cas.client.validation.AbstractCasProtocolUrlBasedTicketValidator.retrieveResponseFromServer(AbstractCasProtocolUrlBasedTicketValidator.java:32) >> at >> org.jasig.cas.client.validation.AbstractUrlBasedTicketValidator.validate(AbstractUrlBasedTicketValidator.java:187) >> at >> edu.tamu.cas.client.blackboard.CASAuthenticator.validate(CASAuthenticator.java:216) >> at >> edu.tamu.cas.client.blackboard.TAMUCASModule.login(TAMUCASModule.java:226) >> at >> com.webct.platform.coreservice.security.authentication.module.ProxyAuthenticationModule.login(ProxyAuthenticationModule.java:135) >> at sun.reflect.GeneratedMethodAccessor229.invoke(Unknown Source) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> at java.lang.reflect.Method.invoke(Method.java:585) >> at javax.security.auth.login.LoginContext.invoke(LoginContext.java:769) >> at >> javax.security.auth.login.LoginContext.access$000(LoginContext.java:186) >> at javax.security.auth.login.LoginContext$4.run(LoginContext.java:683) >> at java.security.AccessController.doPrivileged(Native Method) >> at >> javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680) >> at javax.security.auth.login.LoginContext.login(LoginContext.java:579) >> at >> com.webct.platform.coreservice.security.authentication.common.VistaAgent.authenticate(VistaAgent.java:118) >> at >> com.webct.platform.coreservice.security.authentication.common.VistaAuthenticationManager.getAgent(VistaAuthenticationManager.java:95) >> at >> com.webct.platform.coreservice.security.authentication.common.AuthenticationHandler.performSSOAuthentication(AuthenticationHandler.java:271) >> at >> com.webct.platform.coreservice.security.authentication.common.AuthenticationHandler.handleRequest(AuthenticationHandler.java:109) >> at >> com.webct.platform.coreservice.action.RequestProcessor.process(RequestProcessor.java:171) >> at >> org.apache.struts.action.ActionServlet.process(ActionServlet.java:1164) >> at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:397) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:743) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:856) >> at >> weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:225) >> at >> weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:127) >> at >> weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:283) >> at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:26) >> at >> weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:42) >> at >> com.webct.platform.coreservice.action.CacheControlFilter.doFilter(CacheControlFilter.java:90) >> at >> weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:42) >> at >> com.webct.platform.coreservice.action.RequestCleanupFilter.doFilter(RequestCleanupFilter.java:160) >> at >> weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:42) >> at >> weblogic.servlet.internal.RequestDispatcherImpl.invokeServlet(RequestDispatcherImpl.java:501) >> at >> weblogic.servlet.internal.RequestDispatcherImpl.forward(RequestDispatcherImpl.java:245) >> at >> com.webct.platform.coreservice.action.UrlRewritingServlet.service(UrlRewritingServlet.java:76) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:856) >> at >> weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:225) >> at >> weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:127) >> at >> weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:283) >> at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:26) >> at >> weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:42) >> at >> com.webct.platform.coreservice.action.CacheControlFilter.doFilter(CacheControlFilter.java:90) >> at >> weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:42) >> at >> com.webct.platform.coreservice.action.RequestCleanupFilter.doFilter(RequestCleanupFilter.java:160) >> at >> weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:42) >> at >> weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3212) >> at >> weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321) >> at >> weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121) >> at >> weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:1983) >> at >> weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:1890) >> at >> weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1344) >> at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209) >> at weblogic.work.ExecuteThread.run(ExecuteThread.java:181) >> >> This is using Jasig CAS client 3.1.11, but the line numbers are affected >> by some code modifications added by me in a futile attempt at debugging >> this. >> >> I searched list archives and came back with nothing. There were some >> previously documented issues with keep-alive and SSL. The legacy Yale >> CAS client even added a "Connection: close" HTTP header in an apparent >> attempt to sidestep keep-alive issues. I tried that in the Jasig CAS >> client with no luck. >> >> Has anyone experienced something similar? Sadly, this issue does not >> occur in test or dev environments, so the actual users are affected. :-( >> >> Thanks, >> Adam >> >> > > -- > > Susan Bramhall ([email protected] <mailto:[email protected]>) > Senior Developer, Infrastructure Systems and Architecture > Yale University Information Technology Services (ITS) > 25 Science Park, 150 Munson St, New Haven, CT 06520 > Phone: 203 432 6697 > > -- > You are currently subscribed [email protected] > <mailto:[email protected]> as: [email protected] > To unsubscribe, change settings or access archives, see > http://www.ja-sig.org/wiki/display/JSG/cas-dev -- You are currently subscribed to [email protected] as: [email protected] To unsubscribe, change settings or access archives, see http://www.ja-sig.org/wiki/display/JSG/cas-dev
<<attachment: arybicki.vcf>>
