Ryan,This may be the same problem as the previously reported issue when using JPATicketRegistry under load. This is because the DefaultRegistryCleaner does a periodic pass through the entire registry looking for expired tickets. Due to the limitations of the TicketRegistry and RegistryCleaner interfaces, his happens in a single transaction, and it obviously takes time.
What can you do? One choice would be to write a custom registry cleaner that does not lock the entire ticket table. Instead of using the getTickets() method to get every ticket in the registry, it would perform its own SQL to get all the ticket IDs, and iterate through this list by using getTicket()/isExpired()/deleteTicket(). Just make sure that only one getTicket()/isExpired()/deleteTicket() per transaction is executed.
Another choice would be to disable the ticket registry cleaner completely and write a separate ticket registry purging process outside of CAS. CAS will operate correctly without the ticket registry cleaner, as it will detect the expired tickets when they are presented by a browser. The cleaner needs to keep the database from growing infinitely.
Adam Ryan Andreasen wrote:
I hope someone might be able to shed some light on this problem or let me know if they have experienced similar situations. This is the environment: * 2 Tomcat servers running CAS that are fronted with a hardware load-balancer that guarantees session affinity (the user always goes to the 1st server they hit) * Using a JPATicketRegistry configured to use a MS SQL Server database (for the TGTs and STs)We put this under a real load and things ran smoothly for about 4 hours. Then we started to get deadlock exceptions on one of the CAS servers. Mythought was that the ticket cleaning of one server was locking the table and causing the other server to fail. So we took that server out of the load balancer and undeployed CAS (so that its cleaner would not run). Things again ran smoothly for about 2 hours or so and then the other CAS server had similar (if not the same) deadlock problems. The JPATicketRegistry code looks very straight-forward so I am wondering if SQL Server is the root of these problems. Any help would be greatly appreciated. Below are some ofthe outputs from the cas.log files (sorry for the enormous stack trace). Thanks so much for any help or insight!!--------------------- LOG FILE ENTRY ----------------- 2009-04-01 16:16:34,732 ERROR [org.hibernate.util.JDBCExceptionReporter] - Transaction (Process ID 99) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction. 2009-04-01 16:16:34,732 ERROR [org.hibernate.event.def.AbstractFlushingEventListener] - Could not synchronize database state with session org.hibernate.exception.LockAcquisitionException: could not insert: [org.jasig.cas.ticket.TicketGrantingTicketImpl] at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:82) at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:43) at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2267) at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2660) at org.hibernate.action.EntityInsertAction.execute(EntityInsertAction.java:56) at org.hibernate.engine.ActionQueue.execute(ActionQueue.java:250) at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:234) at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:141) at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:298) at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:27) at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1000) at org.hibernate.impl.SessionImpl.managedFlush(SessionImpl.java:338) at org.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:106) at org.hibernate.ejb.TransactionImpl.commit(TransactionImpl.java:54) at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:456) at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:709) at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:678) at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:321) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:116) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) at $Proxy51.createTicketGrantingTicket(Unknown Source) at org.jasig.cas.web.flow.AuthenticationViaFormAction.submit(AuthenticationViaFormAction.java:107) at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.springframework.webflow.util.DispatchMethodInvoker.invoke(DispatchMethodInvoker.java:99) at org.springframework.webflow.action.MultiAction.doExecute(MultiAction.java:133) at org.springframework.webflow.action.AbstractAction.execute(AbstractAction.java:192) at org.springframework.webflow.engine.AnnotatedAction.execute(AnnotatedAction.java:146) at org.springframework.webflow.engine.ActionExecutor.execute(ActionExecutor.java:59) at org.springframework.webflow.engine.ActionState.doEnter(ActionState.java:156) at org.springframework.webflow.engine.State.enter(State.java:191) at org.springframework.webflow.engine.Transition.execute(Transition.java:212) at org.springframework.webflow.engine.TransitionableState.onEvent(TransitionableState.java:107) at org.springframework.webflow.engine.Flow.onEvent(Flow.java:534) at org.springframework.webflow.engine.impl.RequestControlContextImpl.signalEvent(RequestControlContextImpl.java:205) at org.springframework.webflow.engine.ActionState.doEnter(ActionState.java:161) at org.springframework.webflow.engine.State.enter(State.java:191) at org.springframework.webflow.engine.Transition.execute(Transition.java:212) at org.springframework.webflow.engine.TransitionableState.onEvent(TransitionableState.java:107) at org.springframework.webflow.engine.Flow.onEvent(Flow.java:534) at org.springframework.webflow.engine.impl.RequestControlContextImpl.signalEvent(RequestControlContextImpl.java:205) at org.springframework.webflow.engine.impl.FlowExecutionImpl.signalEvent(FlowExecutionImpl.java:202) at org.springframework.webflow.executor.FlowExecutorImpl.resume(FlowExecutorImpl.java:222) at org.springframework.webflow.executor.support.FlowRequestHandler.handleFlowRequest(FlowRequestHandler.java:111) at org.springframework.webflow.executor.mvc.FlowController.handleRequestInternal(FlowController.java:165) at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153) at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:875) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:807) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:571) at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:511) at javax.servlet.http.HttpServlet.service(HttpServlet.java:637) at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) at org.jasig.cas.web.init.SafeDispatcherServlet.service(SafeDispatcherServlet.java:115) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.inspektr.common.web.ClientInfoThreadLocalFilter.doFilterInternal(ClientInfoThreadLocalFilter.java:48) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286) at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190) at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283) at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:767) at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:697) at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:889) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690) at java.lang.Thread.run(Thread.java:619) Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: Transaction (Process ID 99) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction. at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(Unknown Source) at com.microsoft.sqlserver.jdbc.IOBuffer.processPackets(Unknown Source) at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.getPrepExecResponse(Unknown Source) at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(Unknown Source) at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PreparedStatementExecutionRequest.executeStatement(Unknown Source) at com.microsoft.sqlserver.jdbc.CancelableRequest.execute(Unknown Source) at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeRequest(Unknown Source) at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeUpdate(Unknown Source) at org.hibernate.jdbc.NonBatchingBatcher.addToBatch(NonBatchingBatcher.java:23) at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2247) ... 72 more
begin:vcard fn:Adam Rybicki n:Rybicki;Adam org:Unicon, Inc.;Professional Services adr:Suite 113;;3140 North Arizona Avenue;Chandler;AZ;85225;United States email;internet:[email protected] tel;work:+1-480-558-2400 tel;home:+1-310-265-8286 tel;cell:+1-310-980-2758 x-mozilla-html:FALSE url:http://www.unicon.net/ version:2.1 end:vcard
smime.p7s
Description: S/MIME Cryptographic Signature
