Re: [cas-user] After a month, no tickets created in 4.2.2?
I should clarify, this has been seen in v4.2.5, as that was the latest version of CAS at the time. 4.2.6 doesn't seem to have any commits targeting this issue specifically. I'm definitely gunning for v5 when it drops in GA. For now, I think I'm going to turn on max debug in production to see if I can get anything. On Wednesday, November 2, 2016 at 3:34:53 AM UTC-7, dkopy...@unicon.net wrote: > > 4.2.2, right? One other suggestion would be to get on the latest release > in the 4.2.x series, that is 4.2.6 ... or even get on the v5 bandwagon > > Cheers, > D. > > On Nov 1, 2016, 21:24 -0400, Jeffrey Wong >, > wrote: > > This issue occurred again tonight. Nothing of note in the logs again, with > plenty of jvm memory + disk space left. It just will start redirecting > users to the login page as if a failed login occurred, but without > displaying any error messages. > > Typing in an incorrect password is caught and the error is displayed as > expected with an authentication failed audit. Typing in a correct password > will log a successful authentication, but not generate/validate any service > tickets in the audit. > > This is not related to any LDAP connectivity (this was my reason for > updating in the first place) as I also tested on local password storage > during this outage as well. Swapping ticketing systems does not seem to > have helped. > > Any suggestions of what else I should try? Would getting out of tomcat and > running it under a separate container help? > > On Friday, September 23, 2016 at 3:45:49 PM UTC-7, Jeffrey Wong wrote: >> >> On suggestions from another user with similar issues using JPA, I have >> changed and deployed CAS using a hazelcast ticketing database. >> >> I'll let you know if I have any success with this configuration. >> >> On Thursday, September 22, 2016 at 11:55:16 AM UTC-7, Jeffrey Wong wrote: >>> >>> Hi again, >>> >>> It's been about a month and have regularly screened the JVM memory - it >>> looks fine since the memory bumps, running CAS v4.2.4. >>> >>> However, the server fell over again (using the JPA ticket registry), >>> with the same behavior: upon entering correct credentials, a user is >>> redirected back to the login page rather than logging in. I'm not sure >>> where to go from here to ensure a more reliable service, and would like to >>> hear your input. >>> >>> While digging, I've found three types of exceptions in the logs: >>> >>> 1: deadlocks >>> SEVERE: Servlet.service() for servlet [cas] in context with path [/cas] >>> threw exception [Request processing failed; nested exception is >>> org.springframework.webflow.execution.ActionExecutionException: Exception >>> thrown executing >>> org.jasig.cas.web.flow.GenerateServiceTicketAction@4805bd13 in state >>> 'generateServiceTicket' of flow 'login' -- action execution attributes were >>> 'map[[empty]]'] with root cause >>> com.mysql.jdbc.exceptions.jdbc4.MySQLTransactionRollbackException: >>> Deadlock found when trying to get lock; try restarting transaction >>> [265/9592] >>> at >>> sun.reflect.GeneratedConstructorAccessor127.newInstance(Unknown Source) >>> at >>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) >>> at >>> java.lang.reflect.Constructor.newInstance(Constructor.java:423) >>> at com.mysql.jdbc.Util.handleNewInstance(Util.java:404) >>> at com.mysql.jdbc.Util.getInstance(Util.java:387) >>> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:946) >>> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3878) >>> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3814) >>> at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2478) >>> at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2625) >>> at >>> com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2551) >>> at >>> com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1861) >>> at >>> com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2073) >>> at >>> com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1751) >>> at >>> com.mysql.jdbc.PreparedStatement.executeBatchInternal(PreparedStatement.java:1257) >>> at >>> com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:959) >>> at >>> com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeBatch(NewProxyPreparedStatement.java:2544) >>> >>> 2: badly formatted keys >>> SEVERE: Servlet.service() for servlet [cas] in context with path [/cas] >>> threw exception [Request processing failed; nested exception is >>> org.springframework.webflow.execution.repository.BadlyFormattedFlowExecution >>> KeyException: Badly formatted flow execution key '', the expected format >>> is '_'] with root cause >>> org.spri
Re: [cas-user] After a month, no tickets created in 4.2.2?
4.2.2, right? One other suggestion would be to get on the latest release in the 4.2.x series, that is 4.2.6 ... or even get on the v5 bandwagon Cheers, D. On Nov 1, 2016, 21:24 -0400, Jeffrey Wong , wrote: > This issue occurred again tonight. Nothing of note in the logs again, with > plenty of jvm memory + disk space left. It just will start redirecting users > to the login page as if a failed login occurred, but without displaying any > error messages. > > Typing in an incorrect password is caught and the error is displayed as > expected with an authentication failed audit. Typing in a correct password > will log a successful authentication, but not generate/validate any service > tickets in the audit. > > This is not related to any LDAP connectivity (this was my reason for updating > in the first place) as I also tested on local password storage during this > outage as well. Swapping ticketing systems does not seem to have helped. > > Any suggestions of what else I should try? Would getting out of tomcat and > running it under a separate container help? > > On Friday, September 23, 2016 at 3:45:49 PM UTC-7, Jeffrey Wong wrote: > > On suggestions from another user with similar issues using JPA, I have > > changed and deployed CAS using a hazelcast ticketing database. > > > > I'll let you know if I have any success with this configuration. > > > > On Thursday, September 22, 2016 at 11:55:16 AM UTC-7, Jeffrey Wong wrote: > > > Hi again, > > > > > > It's been about a month and have regularly screened the JVM memory - it > > > looks fine since the memory bumps, running CAS v4.2.4. > > > > > > However, the server fell over again (using the JPA ticket registry), with > > > the same behavior: upon entering correct credentials, a user is > > > redirected back to the login page rather than logging in. I'm not sure > > > where to go from here to ensure a more reliable service, and would like > > > to hear your input. > > > > > > While digging, I've found three types of exceptions in the logs: > > > > > > 1: deadlocks > > > SEVERE: Servlet.service() for servlet [cas] in context with path [/cas] > > > threw exception [Request processing failed; nested exception is > > > org.springframework.webflow.execution.ActionExecutionException: Exception > > > thrown executing > > > org.jasig.cas.web.flow.GenerateServiceTicketAction@4805bd13 in state > > > 'generateServiceTicket' of flow 'login' -- action execution attributes > > > were 'map[[empty]]'] with root cause > > > com.mysql.jdbc.exceptions.jdbc4.MySQLTransactionRollbackException: > > > Deadlock found when trying to get lock; try restarting transaction > > > [265/9592] > > > at sun.reflect.GeneratedConstructorAccessor127.newInstance(Unknown Source) > > > at > > > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > > > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > > > at com.mysql.jdbc.Util.handleNewInstance(Util.java:404) > > > at com.mysql.jdbc.Util.getInstance(Util.java:387) > > > at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:946) > > > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3878) > > > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3814) > > > at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2478) > > > at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2625) > > > at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2551) > > > at > > > com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1861) > > > at > > > com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2073) > > > at > > > com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1751) > > > at > > > com.mysql.jdbc.PreparedStatement.executeBatchInternal(PreparedStatement.java:1257) > > > at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:959) > > > at > > > com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeBatch(NewProxyPreparedStatement.java:2544) > > > > > > > > > 2: badly formatted keys > > > SEVERE: Servlet.service() for servlet [cas] in context with path [/cas] > > > threw exception [Request processing failed; nested exception is > > > org.springframework.webflow.execution.repository.BadlyFormattedFlowExecution > > > KeyException: Badly formatted flow execution key '', the expected format > > > is '_'] with root cause > > > org.springframework.webflow.execution.repository.BadlyFormattedFlowExecutionKeyException: > > > Badly formatted flow execution key '', the expected format is > > > '_' > > > at > > > org.jasig.spring.webflow.plugin.ClientFlowExecutionKey.parse(ClientFlowExecutionKey.java:102) > > > at > > > org.jasig.spring.webflow.plugin.ClientFlowExecutionRepository.parseFlowExecutionKey(ClientFlowExecutionRepository.java:74) > > > at > > > org.springframework.webflow.executor.FlowExecutorImpl.resumeExecution(FlowExecutorImpl.java:164) > > > at > > >
Re: [cas-user] After a month, no tickets created in 4.2.2?
This issue occurred again tonight. Nothing of note in the logs again, with plenty of jvm memory + disk space left. It just will start redirecting users to the login page as if a failed login occurred, but without displaying any error messages. Typing in an incorrect password is caught and the error is displayed as expected with an authentication failed audit. Typing in a correct password will log a successful authentication, but not generate/validate any service tickets in the audit. This is not related to any LDAP connectivity (this was my reason for updating in the first place) as I also tested on local password storage during this outage as well. Swapping ticketing systems does not seem to have helped. Any suggestions of what else I should try? Would getting out of tomcat and running it under a separate container help? On Friday, September 23, 2016 at 3:45:49 PM UTC-7, Jeffrey Wong wrote: > > On suggestions from another user with similar issues using JPA, I have > changed and deployed CAS using a hazelcast ticketing database. > > I'll let you know if I have any success with this configuration. > > On Thursday, September 22, 2016 at 11:55:16 AM UTC-7, Jeffrey Wong wrote: >> >> Hi again, >> >> It's been about a month and have regularly screened the JVM memory - it >> looks fine since the memory bumps, running CAS v4.2.4. >> >> However, the server fell over again (using the JPA ticket registry), with >> the same behavior: upon entering correct credentials, a user is redirected >> back to the login page rather than logging in. I'm not sure where to go >> from here to ensure a more reliable service, and would like to hear your >> input. >> >> While digging, I've found three types of exceptions in the logs: >> >> 1: deadlocks >> SEVERE: Servlet.service() for servlet [cas] in context with path [/cas] >> threw exception [Request processing failed; nested exception is >> org.springframework.webflow.execution.ActionExecutionException: Exception >> thrown executing >> org.jasig.cas.web.flow.GenerateServiceTicketAction@4805bd13 in state >> 'generateServiceTicket' of flow 'login' -- action execution attributes were >> 'map[[empty]]'] with root cause >> com.mysql.jdbc.exceptions.jdbc4.MySQLTransactionRollbackException: >> Deadlock found when trying to get lock; try restarting transaction >> [265/9592] >> at >> sun.reflect.GeneratedConstructorAccessor127.newInstance(Unknown Source) >> at >> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) >> at java.lang.reflect.Constructor.newInstance(Constructor.java:423) >> at com.mysql.jdbc.Util.handleNewInstance(Util.java:404) >> at com.mysql.jdbc.Util.getInstance(Util.java:387) >> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:946) >> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3878) >> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3814) >> at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2478) >> at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2625) >> at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2551) >> at >> com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1861) >> at >> com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2073) >> at >> com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1751) >> at >> com.mysql.jdbc.PreparedStatement.executeBatchInternal(PreparedStatement.java:1257) >> at >> com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:959) >> at >> com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeBatch(NewProxyPreparedStatement.java:2544) >> >> 2: badly formatted keys >> SEVERE: Servlet.service() for servlet [cas] in context with path [/cas] >> threw exception [Request processing failed; nested exception is >> org.springframework.webflow.execution.repository.BadlyFormattedFlowExecution >> KeyException: Badly formatted flow execution key '', the expected format >> is '_'] with root cause >> org.springframework.webflow.execution.repository.BadlyFormattedFlowExecutionKeyException: >> >> Badly formatted flow execution key '', the expected format is >> '_' >> at >> org.jasig.spring.webflow.plugin.ClientFlowExecutionKey.parse(ClientFlowExecutionKey.java:102) >> at >> org.jasig.spring.webflow.plugin.ClientFlowExecutionRepository.parseFlowExecutionKey(ClientFlowExecutionRepository.java:74) >> at >> org.springframework.webflow.executor.FlowExecutorImpl.resumeExecution(FlowExecutorImpl.java:164) >> at >> org.springframework.webflow.mvc.servlet.FlowHandlerAdapter.handle(FlowHandlerAdapter.java:228) >> at >> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:95
Re: [cas-user] After a month, no tickets created in 4.2.2?
On suggestions from another user with similar issues using JPA, I have changed and deployed CAS using a hazelcast ticketing database. I'll let you know if I have any success with this configuration. On Thursday, September 22, 2016 at 11:55:16 AM UTC-7, Jeffrey Wong wrote: > > Hi again, > > It's been about a month and have regularly screened the JVM memory - it > looks fine since the memory bumps, running CAS v4.2.4. > > However, the server fell over again (using the JPA ticket registry), with > the same behavior: upon entering correct credentials, a user is redirected > back to the login page rather than logging in. I'm not sure where to go > from here to ensure a more reliable service, and would like to hear your > input. > > While digging, I've found three types of exceptions in the logs: > > 1: deadlocks > SEVERE: Servlet.service() for servlet [cas] in context with path [/cas] > threw exception [Request processing failed; nested exception is > org.springframework.webflow.execution.ActionExecutionException: Exception > thrown executing > org.jasig.cas.web.flow.GenerateServiceTicketAction@4805bd13 in state > 'generateServiceTicket' of flow 'login' -- action execution attributes were > 'map[[empty]]'] with root cause > com.mysql.jdbc.exceptions.jdbc4.MySQLTransactionRollbackException: > Deadlock found when trying to get lock; try restarting transaction > [265/9592] > at sun.reflect.GeneratedConstructorAccessor127.newInstance(Unknown > Source) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > at com.mysql.jdbc.Util.handleNewInstance(Util.java:404) > at com.mysql.jdbc.Util.getInstance(Util.java:387) > at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:946) > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3878) > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3814) > at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2478) > at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2625) > at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2551) > at > com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1861) > at > com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2073) > at > com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1751) > at > com.mysql.jdbc.PreparedStatement.executeBatchInternal(PreparedStatement.java:1257) > at > com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:959) > at > com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeBatch(NewProxyPreparedStatement.java:2544) > > 2: badly formatted keys > SEVERE: Servlet.service() for servlet [cas] in context with path [/cas] > threw exception [Request processing failed; nested exception is > org.springframework.webflow.execution.repository.BadlyFormattedFlowExecution > KeyException: Badly formatted flow execution key '', the expected format > is '_'] with root cause > org.springframework.webflow.execution.repository.BadlyFormattedFlowExecutionKeyException: > > Badly formatted flow execution key '', the expected format is > '_' > at > org.jasig.spring.webflow.plugin.ClientFlowExecutionKey.parse(ClientFlowExecutionKey.java:102) > at > org.jasig.spring.webflow.plugin.ClientFlowExecutionRepository.parseFlowExecutionKey(ClientFlowExecutionRepository.java:74) > at > org.springframework.webflow.executor.FlowExecutorImpl.resumeExecution(FlowExecutorImpl.java:164) > at > org.springframework.webflow.mvc.servlet.FlowHandlerAdapter.handle(FlowHandlerAdapter.java:228) > at > org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:959) > at > org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:893) > at > org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) > at > org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:646) > at > org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.apereo.cas.security.ResponseHeadersEnforcementFilter.doFilter(ResponseHeadersEnforcementFilter.java:238) > at > org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFi
Re: [cas-user] After a month, no tickets created in 4.2.2?
Hi again, It's been about a month and have regularly screened the JVM memory - it looks fine since the memory bumps, running CAS v4.2.4. However, the server fell over again (using the JPA ticket registry), with the same behavior: upon entering correct credentials, a user is redirected back to the login page rather than logging in. I'm not sure where to go from here to ensure a more reliable service, and would like to hear your input. While digging, I've found three types of exceptions in the logs: 1: deadlocks SEVERE: Servlet.service() for servlet [cas] in context with path [/cas] threw exception [Request processing failed; nested exception is org.springframework.webflow.execution.ActionExecutionException: Exception thrown executing org.jasig.cas.web.flow.GenerateServiceTicketAction@4805bd13 in state 'generateServiceTicket' of flow 'login' -- action execution attributes were 'map[[empty]]'] with root cause com.mysql.jdbc.exceptions.jdbc4.MySQLTransactionRollbackException: Deadlock found when trying to get lock; try restarting transaction [265/9592] at sun.reflect.GeneratedConstructorAccessor127.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.mysql.jdbc.Util.handleNewInstance(Util.java:404) at com.mysql.jdbc.Util.getInstance(Util.java:387) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:946) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3878) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3814) at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2478) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2625) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2551) at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1861) at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2073) at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1751) at com.mysql.jdbc.PreparedStatement.executeBatchInternal(PreparedStatement.java:1257) at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:959) at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeBatch(NewProxyPreparedStatement.java:2544) 2: badly formatted keys SEVERE: Servlet.service() for servlet [cas] in context with path [/cas] threw exception [Request processing failed; nested exception is org.springframework.webflow.execution.repository.BadlyFormattedFlowExecution KeyException: Badly formatted flow execution key '', the expected format is '_'] with root cause org.springframework.webflow.execution.repository.BadlyFormattedFlowExecutionKeyException: Badly formatted flow execution key '', the expected format is '_' at org.jasig.spring.webflow.plugin.ClientFlowExecutionKey.parse(ClientFlowExecutionKey.java:102) at org.jasig.spring.webflow.plugin.ClientFlowExecutionRepository.parseFlowExecutionKey(ClientFlowExecutionRepository.java:74) at org.springframework.webflow.executor.FlowExecutorImpl.resumeExecution(FlowExecutorImpl.java:164) at org.springframework.webflow.mvc.servlet.FlowHandlerAdapter.handle(FlowHandlerAdapter.java:228) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:959) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:893) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872) at javax.servlet.http.HttpServlet.service(HttpServlet.java:646) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apereo.cas.security.ResponseHeadersEnforcementFilter.doFilter(ResponseHeadersEnforcementFilter.java:238) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) 3: constraint violations Sep 22, 2016 9:48:20 AM org.apache.catalina.core.StandardWrapperValve invoke SEVERE: Servlet.service() for servlet [cas] in context with path [/cas] threw exception [Request processing failed; nested exception is org.springframework.webflow.execution.ActionExecutionException: Exception th rown executing org.jasig.cas.web.flow.GenerateServiceTicketAction@4805bd13 in state 'generateServiceTicket' of
Re: [cas-user] After a month, no tickets created in 4.2.2?
Ah, JVM stats. Thanks for that tip: I really wasn't looking at my memory closely enough, and I still had a -Xmx128m set (tomcat7 default in ubuntu). I bumped it to 1g (actually this time, confirmed via /status this time) so I'm hoping this helps resolve. Ah! That should do it. I am surprised you were able to last a few months with that setting. My preference is somewhere between 2g and 4g. /status wasn't working for localhost by default, but this was due to IPv6 on my end. Would you be open to changing the default cas.securityContext.adminpages.ip=(127\.0\.0\.1|0:0:0:0:0:0:0:1) to support IPv6 localhost? No problem. File an issue please. I'm planning on throwing on a lot more memory monitoring near future. I'm assuming this would be a very likely root cause for these sorts off issues - sorry I didn't catch that earlier. Hoping this is the ticket (hurr hurr) - I'll let you know if anything else interesting comes up, but I really appreciate your support! No worries. All I did was to echo back what you already knew. -- You received this message because you are subscribed to the Google Groups "CAS Community" group. To unsubscribe from this group and stop receiving emails from it, send an email to cas-user+unsubscr...@apereo.org. To post to this group, send email to cas-user@apereo.org. Visit this group at https://groups.google.com/a/apereo.org/group/cas-user/. To view this discussion on the web visit https://groups.google.com/a/apereo.org/d/msgid/cas-user/etPan.57bfda0e.2e81feef.291c%40unicon.net. For more options, visit https://groups.google.com/a/apereo.org/d/optout.
Re: [cas-user] After a month, no tickets created in 4.2.2?
Ah, JVM stats. Thanks for that tip: I really wasn't looking at my memory closely enough, and I still had a -Xmx128m set (tomcat7 default in ubuntu). I bumped it to 1g (actually this time, confirmed via /status this time) so I'm hoping this helps resolve. /status wasn't working for localhost by default, but this was due to IPv6 on my end. Would you be open to changing the default cas.securityContext.adminpages.ip=(127\.0\.0\.1|0:0:0:0:0:0:0:1) to support IPv6 localhost? I'm planning on throwing on a lot more memory monitoring near future. I'm assuming this would be a very likely root cause for these sorts off issues - sorry I didn't catch that earlier. Hoping this is the ticket (hurr hurr) - I'll let you know if anything else interesting comes up, but I really appreciate your support! On Thursday, August 25, 2016 at 10:40:07 AM UTC-7, Misagh Moayyed wrote: > > Thanks for the submission. > > That should suffice. Watch your server logs as well, and keep an eye up > JVM performance and stats. > > -- > Misagh > > From: Jeffrey Wong > Reply: Jeffrey Wong > Date: August 25, 2016 at 9:43:21 AM > To: CAS Community > Cc: mmoa...@unicon.net > Subject: Re: [cas-user] After a month, no tickets created in 4.2.2? > > OK, thanks for the confirmation. I sent in a ticket about it. > > As for the original issue, I'll continue to monitor to see if I detect > anything on my end. Aside from a full debug on the jasig.cas namespace, > would there be any other library/namespace that would be beneficial to > include in the logs? > > On Wednesday, August 24, 2016 at 10:34:11 PM UTC-7, Misagh Moayyed wrote: >> >> You’re right. There seems to be a problem, but it’s not a cleanup >> problem. Your logs actually show all 10 tickets are being removed >> correctly. If you scan through, you’ll find “Removing ticket [TGT-…]” 5 >> times and you’ll find "Removed ticket [ST-…” 5 times. >> >> The count is inconsistent; because orphaned STs are counted and deleted >> separately. STs that are attached to a TGT are not counted, yet are cleaned >> when the TGT is removed. So in effect, you have 5 TGTs removed, 2 orphaned >> STs (counting up to 7 in the final output) and other STs are removed as >> part of those TGTs which you see the removal in the logs but not in the >> final count of them. >> >> Do file a bug please. >> >> Back to the original issue; the cleanup process seems all correct. The >> issue lies somewhere else. >> >> -- >> Misagh >> >> From: Jeffrey Wong >> Reply: Jeffrey Wong >> Date: August 24, 2016 at 2:20:36 PM >> To: CAS Community >> Cc: mmoa...@unicon.net >> Subject: Re: [cas-user] After a month, no tickets created in 4.2.2? >> >> I'm still getting inconsistent ST cleanup summaries using 4.2.5-SNAPSHOT >> with the default in memory cleaner - See the attached log. >> >> Note that all service tickets are found for cleanup, but only two here >> have logged "Attempting to retrieve ticket". It appears in this version >> that all service tickets are found, but only are two are 'expired' at the >> point where the TGTs expire. >> >> Again, this is despite having st.timeToKillInSeconds=600. The default >> registry does 2 minutes pass per clean - the STs should all be expiring >> when the TGTs expire (if I'm understanding correctly) - otherwise, they >> should expire when timeToKillInSeconds passes. In this case 10 minutes. >> >> Assuming the STs expire on their own, I waited 10 minutes: >> >> grep "expired tickets found and removed." cas.log: >> >> 2016-08-24 16:47:21,814 INFO >> [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 7 expired tickets >> found and removed. <- 5 TGTs + 2 STs, the last line of the attached log >> 2016-08-24 16:49:21,714 INFO >> [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets >> found and removed. >> 2016-08-24 16:51:21,718 INFO >> [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets >> found and removed. >> 2016-08-24 16:53:21,723 INFO >> [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets >> found and removed. >> 2016-08-24 16:55:21,715 INFO >> [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets >> found and removed. >> 2016-08-24 16:57:21,712 INFO >> [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets >> found and removed. >> 2016-08-24 16:59
Re: [cas-user] After a month, no tickets created in 4.2.2?
Thanks for the submission. That should suffice. Watch your server logs as well, and keep an eye up JVM performance and stats. -- Misagh From: Jeffrey Wong Reply: Jeffrey Wong Date: August 25, 2016 at 9:43:21 AM To: CAS Community Cc: mmoay...@unicon.net Subject: Re: [cas-user] After a month, no tickets created in 4.2.2? OK, thanks for the confirmation. I sent in a ticket about it. As for the original issue, I'll continue to monitor to see if I detect anything on my end. Aside from a full debug on the jasig.cas namespace, would there be any other library/namespace that would be beneficial to include in the logs? On Wednesday, August 24, 2016 at 10:34:11 PM UTC-7, Misagh Moayyed wrote: You’re right. There seems to be a problem, but it’s not a cleanup problem. Your logs actually show all 10 tickets are being removed correctly. If you scan through, you’ll find “Removing ticket [TGT-…]” 5 times and you’ll find "Removed ticket [ST-…” 5 times. The count is inconsistent; because orphaned STs are counted and deleted separately. STs that are attached to a TGT are not counted, yet are cleaned when the TGT is removed. So in effect, you have 5 TGTs removed, 2 orphaned STs (counting up to 7 in the final output) and other STs are removed as part of those TGTs which you see the removal in the logs but not in the final count of them. Do file a bug please. Back to the original issue; the cleanup process seems all correct. The issue lies somewhere else. -- Misagh From: Jeffrey Wong Reply: Jeffrey Wong Date: August 24, 2016 at 2:20:36 PM To: CAS Community Cc: mmoa...@unicon.net Subject: Re: [cas-user] After a month, no tickets created in 4.2.2? I'm still getting inconsistent ST cleanup summaries using 4.2.5-SNAPSHOT with the default in memory cleaner - See the attached log. Note that all service tickets are found for cleanup, but only two here have logged "Attempting to retrieve ticket". It appears in this version that all service tickets are found, but only are two are 'expired' at the point where the TGTs expire. Again, this is despite having st.timeToKillInSeconds=600. The default registry does 2 minutes pass per clean - the STs should all be expiring when the TGTs expire (if I'm understanding correctly) - otherwise, they should expire when timeToKillInSeconds passes. In this case 10 minutes. Assuming the STs expire on their own, I waited 10 minutes: grep "expired tickets found and removed." cas.log: 2016-08-24 16:47:21,814 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 7 expired tickets found and removed. <- 5 TGTs + 2 STs, the last line of the attached log 2016-08-24 16:49:21,714 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 16:51:21,718 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 16:53:21,723 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 16:55:21,715 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 16:57:21,712 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 16:59:21,715 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 17:01:21,717 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 17:03:21,711 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 17:05:21,711 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 17:07:21,711 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 17:09:21,711 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 17:11:21,713 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. The 3 floating STs should have expired somewhere, but don't appear to be. On Tuesday, August 23, 2016 at 8:54:29 PM UTC-7, Misagh Moayyed wrote: Cool. Thanks for the report. I can run some tests on this, and it would be more reassuring if you were to switch to 4.2.5-SNAPSHOT and test the latest patch that William has put together that handles this sort of thing. If the problem persists, we could further review it before the official 4.2.5, which is in about 10 days. In 4.2.5, the cleanup logic is shared by all registry types. So the behavior is made consistent across all where the task of ticket removal is then delegated to the individual registry that knows how to talk to the source. I suggest that you run your same batch of tests against that baseline, and if you
Re: [cas-user] After a month, no tickets created in 4.2.2?
OK, thanks for the confirmation. I sent in a ticket about it. As for the original issue, I'll continue to monitor to see if I detect anything on my end. Aside from a full debug on the jasig.cas namespace, would there be any other library/namespace that would be beneficial to include in the logs? On Wednesday, August 24, 2016 at 10:34:11 PM UTC-7, Misagh Moayyed wrote: > > You’re right. There seems to be a problem, but it’s not a cleanup problem. > Your logs actually show all 10 tickets are being removed correctly. If > you scan through, you’ll find “Removing ticket [TGT-…]” 5 times and you’ll > find "Removed ticket [ST-…” 5 times. > > The count is inconsistent; because orphaned STs are counted and deleted > separately. STs that are attached to a TGT are not counted, yet are cleaned > when the TGT is removed. So in effect, you have 5 TGTs removed, 2 orphaned > STs (counting up to 7 in the final output) and other STs are removed as > part of those TGTs which you see the removal in the logs but not in the > final count of them. > > Do file a bug please. > > Back to the original issue; the cleanup process seems all correct. The > issue lies somewhere else. > > -- > Misagh > > From: Jeffrey Wong > Reply: Jeffrey Wong > Date: August 24, 2016 at 2:20:36 PM > To: CAS Community > Cc: mmoa...@unicon.net > Subject: Re: [cas-user] After a month, no tickets created in 4.2.2? > > I'm still getting inconsistent ST cleanup summaries using 4.2.5-SNAPSHOT > with the default in memory cleaner - See the attached log. > > Note that all service tickets are found for cleanup, but only two here > have logged "Attempting to retrieve ticket". It appears in this version > that all service tickets are found, but only are two are 'expired' at the > point where the TGTs expire. > > Again, this is despite having st.timeToKillInSeconds=600. The default > registry does 2 minutes pass per clean - the STs should all be expiring > when the TGTs expire (if I'm understanding correctly) - otherwise, they > should expire when timeToKillInSeconds passes. In this case 10 minutes. > > Assuming the STs expire on their own, I waited 10 minutes: > > grep "expired tickets found and removed." cas.log: > > 2016-08-24 16:47:21,814 INFO > [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 7 expired tickets > found and removed. <- 5 TGTs + 2 STs, the last line of the attached log > 2016-08-24 16:49:21,714 INFO > [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets > found and removed. > 2016-08-24 16:51:21,718 INFO > [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets > found and removed. > 2016-08-24 16:53:21,723 INFO > [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets > found and removed. > 2016-08-24 16:55:21,715 INFO > [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets > found and removed. > 2016-08-24 16:57:21,712 INFO > [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets > found and removed. > 2016-08-24 16:59:21,715 INFO > [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets > found and removed. > 2016-08-24 17:01:21,717 INFO > [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets > found and removed. > 2016-08-24 17:03:21,711 INFO > [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets > found and removed. > 2016-08-24 17:05:21,711 INFO > [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets > found and removed. > 2016-08-24 17:07:21,711 INFO > [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets > found and removed. > 2016-08-24 17:09:21,711 INFO > [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets > found and removed. > 2016-08-24 17:11:21,713 INFO > [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets > found and removed. > > The 3 floating STs should have expired somewhere, but don't appear to be. > > On Tuesday, August 23, 2016 at 8:54:29 PM UTC-7, Misagh Moayyed wrote: >> >> Cool. Thanks for the report. I can run some tests on this, and it would >> be more reassuring if you were to switch to 4.2.5-SNAPSHOT and test the >> latest patch that William has put together that handles this sort of thing. >> If the problem persists, we could further review it before the official >> 4.2.5, which is in about 10 days. >> >> In 4.2.5, the cleanup logic is shared by all registry types. So the >> behavior is made consistent across all where the task o
Re: [cas-user] After a month, no tickets created in 4.2.2?
You’re right. There seems to be a problem, but it’s not a cleanup problem. Your logs actually show all 10 tickets are being removed correctly. If you scan through, you’ll find “Removing ticket [TGT-…]” 5 times and you’ll find "Removed ticket [ST-…” 5 times. The count is inconsistent; because orphaned STs are counted and deleted separately. STs that are attached to a TGT are not counted, yet are cleaned when the TGT is removed. So in effect, you have 5 TGTs removed, 2 orphaned STs (counting up to 7 in the final output) and other STs are removed as part of those TGTs which you see the removal in the logs but not in the final count of them. Do file a bug please. Back to the original issue; the cleanup process seems all correct. The issue lies somewhere else. -- Misagh From: Jeffrey Wong Reply: Jeffrey Wong Date: August 24, 2016 at 2:20:36 PM To: CAS Community Cc: mmoay...@unicon.net Subject: Re: [cas-user] After a month, no tickets created in 4.2.2? I'm still getting inconsistent ST cleanup summaries using 4.2.5-SNAPSHOT with the default in memory cleaner - See the attached log. Note that all service tickets are found for cleanup, but only two here have logged "Attempting to retrieve ticket". It appears in this version that all service tickets are found, but only are two are 'expired' at the point where the TGTs expire. Again, this is despite having st.timeToKillInSeconds=600. The default registry does 2 minutes pass per clean - the STs should all be expiring when the TGTs expire (if I'm understanding correctly) - otherwise, they should expire when timeToKillInSeconds passes. In this case 10 minutes. Assuming the STs expire on their own, I waited 10 minutes: grep "expired tickets found and removed." cas.log: 2016-08-24 16:47:21,814 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 7 expired tickets found and removed. <- 5 TGTs + 2 STs, the last line of the attached log 2016-08-24 16:49:21,714 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 16:51:21,718 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 16:53:21,723 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 16:55:21,715 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 16:57:21,712 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 16:59:21,715 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 17:01:21,717 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 17:03:21,711 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 17:05:21,711 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 17:07:21,711 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 17:09:21,711 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 17:11:21,713 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. The 3 floating STs should have expired somewhere, but don't appear to be. On Tuesday, August 23, 2016 at 8:54:29 PM UTC-7, Misagh Moayyed wrote: Cool. Thanks for the report. I can run some tests on this, and it would be more reassuring if you were to switch to 4.2.5-SNAPSHOT and test the latest patch that William has put together that handles this sort of thing. If the problem persists, we could further review it before the official 4.2.5, which is in about 10 days. In 4.2.5, the cleanup logic is shared by all registry types. So the behavior is made consistent across all where the task of ticket removal is then delegated to the individual registry that knows how to talk to the source. I suggest that you run your same batch of tests against that baseline, and if you still see the issue, please post back. -- Misagh From: Jeffrey Wong Reply: Jeffrey Wong Date: August 23, 2016 at 2:33:34 PM To: CAS Community Cc: mmoa...@unicon.net Subject: Re: [cas-user] After a month, no tickets created in 4.2.2? Also to note, switching to JPA under the same circumstances, I can confirm that all 10 tickets are consistently reporting in as cleaned (ran over multiple tests). Definitely seems like there's something strange with the in memory ticketing cleaner. On Tuesday, August 23, 2016 at 2:04:51 PM UTC-7, Jeffrey Wong wrote: Ah, looks to be that the PT fix won't affect my issues then - Let me clarify how I'm doing my testing in that case: I'm creating
Re: [cas-user] After a month, no tickets created in 4.2.2?
I'm still getting inconsistent ST cleanup summaries using 4.2.5-SNAPSHOT with the default in memory cleaner - See the attached log. Note that all service tickets are found for cleanup, but only two here have logged "Attempting to retrieve ticket". It appears in this version that all service tickets are found, but only are two are 'expired' at the point where the TGTs expire. Again, this is despite having st.timeToKillInSeconds=600. The default registry does 2 minutes pass per clean - the STs should all be expiring when the TGTs expire (if I'm understanding correctly) - otherwise, they should expire when timeToKillInSeconds passes. In this case 10 minutes. Assuming the STs expire on their own, I waited 10 minutes: grep "expired tickets found and removed." cas.log: 2016-08-24 16:47:21,814 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 7 expired tickets found and removed. <- 5 TGTs + 2 STs, the last line of the attached log 2016-08-24 16:49:21,714 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 16:51:21,718 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 16:53:21,723 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 16:55:21,715 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 16:57:21,712 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 16:59:21,715 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 17:01:21,717 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 17:03:21,711 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 17:05:21,711 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 17:07:21,711 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 17:09:21,711 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-24 17:11:21,713 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. The 3 floating STs should have expired somewhere, but don't appear to be. On Tuesday, August 23, 2016 at 8:54:29 PM UTC-7, Misagh Moayyed wrote: > > Cool. Thanks for the report. I can run some tests on this, and it would be > more reassuring if you were to switch to 4.2.5-SNAPSHOT and test the latest > patch that William has put together that handles this sort of thing. If > the problem persists, we could further review it before the official 4.2.5, > which is in about 10 days. > > In 4.2.5, the cleanup logic is shared by all registry types. So the > behavior is made consistent across all where the task of ticket removal is > then delegated to the individual registry that knows how to talk to the > source. I suggest that you run your same batch of tests against that > baseline, and if you still see the issue, please post back. > > -- > Misagh > > From: Jeffrey Wong > Reply: Jeffrey Wong > Date: August 23, 2016 at 2:33:34 PM > To: CAS Community > Cc: mmoa...@unicon.net > Subject: Re: [cas-user] After a month, no tickets created in 4.2.2? > > Also to note, switching to JPA under the same circumstances, I can confirm > that all 10 tickets are consistently reporting in as cleaned (ran over > multiple tests). Definitely seems like there's something strange with the > in memory ticketing cleaner. > > On Tuesday, August 23, 2016 at 2:04:51 PM UTC-7, Jeffrey Wong wrote: >> >> Ah, looks to be that the PT fix won't affect my issues then - Let me >> clarify how I'm doing my testing in that case: >> >> I'm creating a new session (new cookie map) with every login, over the >> same user. I'm scraping the lt, execution, and _eventId off of the initial >> payload, and making a post request with those parameters with >> username+password. >> >> In the logs, each login creates a new TGT (since I'm not sharing cookies >> per request, I'll have a unique TGT per login) and a new ST. I've turned on >> debugs and can confirm that with a series of 5 logins, 5 TGT and 5 STs are >> created. I turned st.timeToKillInSeconds way down (10) to ensure cleanup of >> all tickets on the next cleaning run. In this test case above, I would >> expect a total of 10 tickets to be cleaned up on the next cleaner run. >> >> Durin
Re: [cas-user] After a month, no tickets created in 4.2.2?
Cool. Thanks for the report. I can run some tests on this, and it would be more reassuring if you were to switch to 4.2.5-SNAPSHOT and test the latest patch that William has put together that handles this sort of thing. If the problem persists, we could further review it before the official 4.2.5, which is in about 10 days. In 4.2.5, the cleanup logic is shared by all registry types. So the behavior is made consistent across all where the task of ticket removal is then delegated to the individual registry that knows how to talk to the source. I suggest that you run your same batch of tests against that baseline, and if you still see the issue, please post back. -- Misagh From: Jeffrey Wong Reply: Jeffrey Wong Date: August 23, 2016 at 2:33:34 PM To: CAS Community Cc: mmoay...@unicon.net Subject: Re: [cas-user] After a month, no tickets created in 4.2.2? Also to note, switching to JPA under the same circumstances, I can confirm that all 10 tickets are consistently reporting in as cleaned (ran over multiple tests). Definitely seems like there's something strange with the in memory ticketing cleaner. On Tuesday, August 23, 2016 at 2:04:51 PM UTC-7, Jeffrey Wong wrote: Ah, looks to be that the PT fix won't affect my issues then - Let me clarify how I'm doing my testing in that case: I'm creating a new session (new cookie map) with every login, over the same user. I'm scraping the lt, execution, and _eventId off of the initial payload, and making a post request with those parameters with username+password. In the logs, each login creates a new TGT (since I'm not sharing cookies per request, I'll have a unique TGT per login) and a new ST. I've turned on debugs and can confirm that with a series of 5 logins, 5 TGT and 5 STs are created. I turned st.timeToKillInSeconds way down (10) to ensure cleanup of all tickets on the next cleaning run. In this test case above, I would expect a total of 10 tickets to be cleaned up on the next cleaner run. During cleanup, I appear to be missing a count of a ST or two. It occasionally claims between 7 and 9 tickets cleaned. I've counted all 5 TGTs, but the STs appear to only occasionally be cleaned up fully. To ensure that STs aren't expiring on their own somehow and are strictly expiring as children of TGTs, I have the following settings: tgt.timeToKillInSeconds=10 st.timeToKillInSeconds=600 It's my understanding that when a TGT gets cleaned, that all STs associated with that TGT are also cleaned up as well, but it doesn't appear that this is the case. My worry is that there's an orphaned ST in memory somewhere still floating around memory, which would be OK if I were ticketing through ORM, as I could audit active tickets directly in sql, or remove them myself. I think in the meantime, I'll move production ticketing to a (slower, but more easily auditable) ORM solution. I wanted to let you know what I'm seeing on my end in case this is a weird edge case in the cleaner. On Monday, August 22, 2016 at 10:13:38 PM UTC-7, Misagh Moayyed wrote: I hate to say this, but it depends! It depends what you mean by login, and it depends what you mean by 10 :) Before we start to discuss the answer to the ultimate question of life, the universe, and everything let me explode this a bit. When you log into CAS, you get a TGT. You get an SSO session. That TGT remains alive so long you don’t log out and so long its expiration policy says it should live. For every application you log into, you will get an ST. The application ideally keeps track of the user session so it wouldn’t have to ask for more STs every time you refresh its page for instance. So: If you log in 10 times to 10 different apps, you get 1 TGT and 10 STs. If the act of logging in requires you to present credentials every time, (i.e. renew=true) then that’s still in the end 1 TGTs and 10 STs active and legible for clean up, because every time you generate a new TGT, the old one is immediately killed and destroyed and it will not show up in the clean up log. The cleanup process cleans expired tickets. Regardless of the ticket type. Doesn’t matter of it’s a TGT, ST, OC, etc. There are many many other types. All you have to remember is, if it’s expired, it gets removed at certain intervals. The only exception to this rule is, proxy-tickets are not removed by the clean up process when you “logout” forcefully, and this something that is fixed in the next patch release, thanks to William. But you’re not using PTs, so... -- Misagh From: Jeffrey Wong Reply: Jeffrey Wong Date: August 22, 2016 at 3:50:03 PM To: CAS Community Cc: mmoa...@unicon.net Subject: Re: [cas-user] After a month, no tickets created in 4.2.2? I am not making use of proxy granting tickets, but I'll report back if it's still an issue once 2.2.5 drops. In the version that I have currently deployed, the cou
Re: [cas-user] After a month, no tickets created in 4.2.2?
Also to note, switching to JPA under the same circumstances, I can confirm that all 10 tickets are consistently reporting in as cleaned (ran over multiple tests). Definitely seems like there's something strange with the in memory ticketing cleaner. On Tuesday, August 23, 2016 at 2:04:51 PM UTC-7, Jeffrey Wong wrote: > > Ah, looks to be that the PT fix won't affect my issues then - Let me > clarify how I'm doing my testing in that case: > > I'm creating a new session (new cookie map) with every login, over the > same user. I'm scraping the lt, execution, and _eventId off of the initial > payload, and making a post request with those parameters with > username+password. > > In the logs, each login creates a new TGT (since I'm not sharing cookies > per request, I'll have a unique TGT per login) and a new ST. I've turned on > debugs and can confirm that with a series of 5 logins, 5 TGT and 5 STs are > created. I turned st.timeToKillInSeconds way down (10) to ensure cleanup of > all tickets on the next cleaning run. In this test case above, I would > expect a total of 10 tickets to be cleaned up on the next cleaner run. > > During cleanup, I appear to be missing a count of a ST or two. It > occasionally claims between 7 and 9 tickets cleaned. I've counted all 5 > TGTs, but the STs appear to only occasionally be cleaned up fully. > > To ensure that STs aren't expiring on their own somehow and are strictly > expiring as children of TGTs, I have the following settings: > tgt.timeToKillInSeconds=10 > st.timeToKillInSeconds=600 > > It's my understanding that when a TGT gets cleaned, that all STs > associated with that TGT are also cleaned up as well, but it doesn't appear > that this is the case. > > My worry is that there's an orphaned ST in memory somewhere still floating > around memory, which would be OK if I were ticketing through ORM, as I > could audit active tickets directly in sql, or remove them myself. I think > in the meantime, I'll move production ticketing to a (slower, but more > easily auditable) ORM solution. I wanted to let you know what I'm seeing on > my end in case this is a weird edge case in the cleaner. > > On Monday, August 22, 2016 at 10:13:38 PM UTC-7, Misagh Moayyed wrote: >> >> I hate to say this, but it depends! It depends what you mean by login, >> and it depends what you mean by 10 :) >> >> Before we start to discuss the answer to the ultimate question of life, >> the universe, and everything let me explode this a bit. >> >> When you log into CAS, you get a TGT. You get an SSO session. That TGT >> remains alive so long you don’t log out and so long its expiration policy >> says it should live. For every application you log into, you will get an >> ST. The application ideally keeps track of the user session so it wouldn’t >> have to ask for more STs every time you refresh its page for instance. So: >> >> If you log in 10 times to 10 different apps, you get 1 TGT and 10 STs. If >> the act of logging in requires you to present credentials every time, (i.e. >> renew=true) then that’s still in the end 1 TGTs and 10 STs active and >> legible for clean up, because every time you generate a new TGT, the old >> one is immediately killed and destroyed and it will not show up in the >> clean up log. >> >> The cleanup process cleans expired tickets. Regardless of the ticket >> type. Doesn’t matter of it’s a TGT, ST, OC, etc. There are many many other >> types. All you have to remember is, if it’s expired, it gets removed at >> certain intervals. The only exception to this rule is, proxy-tickets are >> not removed by the clean up process when you “logout” forcefully, and this >> something that is fixed in the next patch release, thanks to William. But >> you’re not using PTs, so... >> >> -- >> Misagh >> >> From: Jeffrey Wong >> Reply: Jeffrey Wong >> Date: August 22, 2016 at 3:50:03 PM >> To: CAS Community >> Cc: mmoa...@unicon.net >> Subject: Re: [cas-user] After a month, no tickets created in 4.2.2? >> >> I am not making use of proxy granting tickets, but I'll report back if >> it's still an issue once 2.2.5 drops. >> >> In the version that I have currently deployed, the counts on the tickets >> themselves seem strange to me - When I log in 10 times, that's 10 TGTs and >> 10 STs, correct? So I should expect 20 tickets to be cleaned up, rather >> than 15. If it's only counting TGTs, then I should have 10 tickets total. >> >> I've turned on debugging, and
Re: [cas-user] After a month, no tickets created in 4.2.2?
Ah, looks to be that the PT fix won't affect my issues then - Let me clarify how I'm doing my testing in that case: I'm creating a new session (new cookie map) with every login, over the same user. I'm scraping the lt, execution, and _eventId off of the initial payload, and making a post request with those parameters with username+password. In the logs, each login creates a new TGT (since I'm not sharing cookies per request, I'll have a unique TGT per login) and a new ST. I've turned on debugs and can confirm that with a series of 5 logins, 5 TGT and 5 STs are created. I turned st.timeToKillInSeconds way down (10) to ensure cleanup of all tickets on the next cleaning run. In this test case above, I would expect a total of 10 tickets to be cleaned up on the next cleaner run. During cleanup, I appear to be missing a count of a ST or two. It occasionally claims between 7 and 9 tickets cleaned. I've counted all 5 TGTs, but the STs appear to only occasionally be cleaned up fully. To ensure that STs aren't expiring on their own somehow and are strictly expiring as children of TGTs, I have the following settings: tgt.timeToKillInSeconds=10 st.timeToKillInSeconds=600 It's my understanding that when a TGT gets cleaned, that all STs associated with that TGT are also cleaned up as well, but it doesn't appear that this is the case. My worry is that there's an orphaned ST in memory somewhere still floating around memory, which would be OK if I were ticketing through ORM, as I could audit active tickets directly in sql, or remove them myself. I think in the meantime, I'll move production ticketing to a (slower, but more easily auditable) ORM solution. I wanted to let you know what I'm seeing on my end in case this is a weird edge case in the cleaner. On Monday, August 22, 2016 at 10:13:38 PM UTC-7, Misagh Moayyed wrote: > > I hate to say this, but it depends! It depends what you mean by login, and > it depends what you mean by 10 :) > > Before we start to discuss the answer to the ultimate question of life, > the universe, and everything let me explode this a bit. > > When you log into CAS, you get a TGT. You get an SSO session. That TGT > remains alive so long you don’t log out and so long its expiration policy > says it should live. For every application you log into, you will get an > ST. The application ideally keeps track of the user session so it wouldn’t > have to ask for more STs every time you refresh its page for instance. So: > > If you log in 10 times to 10 different apps, you get 1 TGT and 10 STs. If > the act of logging in requires you to present credentials every time, (i.e. > renew=true) then that’s still in the end 1 TGTs and 10 STs active and > legible for clean up, because every time you generate a new TGT, the old > one is immediately killed and destroyed and it will not show up in the > clean up log. > > The cleanup process cleans expired tickets. Regardless of the ticket type. > Doesn’t matter of it’s a TGT, ST, OC, etc. There are many many other types. > All you have to remember is, if it’s expired, it gets removed at certain > intervals. The only exception to this rule is, proxy-tickets are not > removed by the clean up process when you “logout” forcefully, and this > something that is fixed in the next patch release, thanks to William. But > you’re not using PTs, so... > > -- > Misagh > > From: Jeffrey Wong > Reply: Jeffrey Wong > Date: August 22, 2016 at 3:50:03 PM > To: CAS Community > Cc: mmoa...@unicon.net > Subject: Re: [cas-user] After a month, no tickets created in 4.2.2? > > I am not making use of proxy granting tickets, but I'll report back if > it's still an issue once 2.2.5 drops. > > In the version that I have currently deployed, the counts on the tickets > themselves seem strange to me - When I log in 10 times, that's 10 TGTs and > 10 STs, correct? So I should expect 20 tickets to be cleaned up, rather > than 15. If it's only counting TGTs, then I should have 10 tickets total. > > I've turned on debugging, and will be monitoring this issue for when it > happens again, and let you know what I see in the logs. Meanwhile, can you > confirm or correct my understanding of the current ticket cleanup log? It > looks pretty strange from my end. > > On Thursday, July 21, 2016 at 11:24:12 PM UTC-7, Misagh Moayyed wrote: >> >> All expired tickets are removed by the cleaner, regardless of type. Come >> to think of it, do you use CAS for its proxy authentication features? That >> *might* have something to do with it, if you do. >> >> I personally don’t know if I’d recommend switching, because I don’t know >> what the problem is. General
Re: [cas-user] After a month, no tickets created in 4.2.2?
I hate to say this, but it depends! It depends what you mean by login, and it depends what you mean by 10 :) Before we start to discuss the answer to the ultimate question of life, the universe, and everything let me explode this a bit. When you log into CAS, you get a TGT. You get an SSO session. That TGT remains alive so long you don’t log out and so long its expiration policy says it should live. For every application you log into, you will get an ST. The application ideally keeps track of the user session so it wouldn’t have to ask for more STs every time you refresh its page for instance. So: If you log in 10 times to 10 different apps, you get 1 TGT and 10 STs. If the act of logging in requires you to present credentials every time, (i.e. renew=true) then that’s still in the end 1 TGTs and 10 STs active and legible for clean up, because every time you generate a new TGT, the old one is immediately killed and destroyed and it will not show up in the clean up log. The cleanup process cleans expired tickets. Regardless of the ticket type. Doesn’t matter of it’s a TGT, ST, OC, etc. There are many many other types. All you have to remember is, if it’s expired, it gets removed at certain intervals. The only exception to this rule is, proxy-tickets are not removed by the clean up process when you “logout” forcefully, and this something that is fixed in the next patch release, thanks to William. But you’re not using PTs, so... -- Misagh From: Jeffrey Wong Reply: Jeffrey Wong Date: August 22, 2016 at 3:50:03 PM To: CAS Community Cc: mmoay...@unicon.net Subject: Re: [cas-user] After a month, no tickets created in 4.2.2? I am not making use of proxy granting tickets, but I'll report back if it's still an issue once 2.2.5 drops. In the version that I have currently deployed, the counts on the tickets themselves seem strange to me - When I log in 10 times, that's 10 TGTs and 10 STs, correct? So I should expect 20 tickets to be cleaned up, rather than 15. If it's only counting TGTs, then I should have 10 tickets total. I've turned on debugging, and will be monitoring this issue for when it happens again, and let you know what I see in the logs. Meanwhile, can you confirm or correct my understanding of the current ticket cleanup log? It looks pretty strange from my end. On Thursday, July 21, 2016 at 11:24:12 PM UTC-7, Misagh Moayyed wrote: All expired tickets are removed by the cleaner, regardless of type. Come to think of it, do you use CAS for its proxy authentication features? That *might* have something to do with it, if you do. I personally don’t know if I’d recommend switching, because I don’t know what the problem is. Generally, switching to something more robust and distributed is a good idea, but if the problem is something else, it will simply repeat and your new registry will have done nothing to fix it. I would instead turn up logs, load test as much as possible and keep it running under test for some time. Observe. -- Misagh From: Jeffrey Wong Reply: Jeffrey Wong Date: July 20, 2016 at 3:27:17 PM To: CAS Community Cc: mmoa...@unicon.net Subject: Re: [cas-user] After a month, no tickets created in 4.2.2? Grepping through what I have, I can confirm that the tickets are being removed, as I do have log statements like the following: catalina.out.2.gz:2016-07-08 22:19:31,948 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - <1 expired tickets found and removed.> catalina.out.2.gz:2016-07-08 22:21:32,004 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - <2 expired tickets found and removed.> I'm using the default in memory registry, the default tgt.maxTimeToLiveInSeconds + tgt.timeToKillInSeconds. st.timeToKillInSeconds=60. Other pretty bare bones defaults: Would you recommend perhaps a different ticket registry at this point perhaps? I don't think I'm hitting the maximum amount of tickets that the default ticket registry can hold by any means, since the maximum amount of tickets I'm seeing expire is 20. Mostly it's between 0-2 tickets that are expired, so I very much doubt the tickets being the memory bottleneck. --- Before I posted the above, I dug a little and noticed a strange ~200 tickets being cleaned up a bit before the issue (an uptick in cleanup tickets). Perhaps movi
Re: [cas-user] After a month, no tickets created in 4.2.2?
I am not making use of proxy granting tickets, but I'll report back if it's still an issue once 2.2.5 drops. In the version that I have currently deployed, the counts on the tickets themselves seem strange to me - When I log in 10 times, that's 10 TGTs and 10 STs, correct? So I should expect 20 tickets to be cleaned up, rather than 15. If it's only counting TGTs, then I should have 10 tickets total. I've turned on debugging, and will be monitoring this issue for when it happens again, and let you know what I see in the logs. Meanwhile, can you confirm or correct my understanding of the current ticket cleanup log? It looks pretty strange from my end. On Thursday, July 21, 2016 at 11:24:12 PM UTC-7, Misagh Moayyed wrote: > > All expired tickets are removed by the cleaner, regardless of type. Come > to think of it, do you use CAS for its proxy authentication features? That > *might* have something to do with it, if you do. > > I personally don’t know if I’d recommend switching, because I don’t know > what the problem is. Generally, switching to something more robust and > distributed is a good idea, but if the problem is something else, it will > simply repeat and your new registry will have done nothing to fix it. I > would instead turn up logs, load test as much as possible and keep it > running under test for some time. Observe. > > -- > Misagh > > From: Jeffrey Wong > Reply: Jeffrey Wong > Date: July 20, 2016 at 3:27:17 PM > To: CAS Community > Cc: mmoa...@unicon.net > Subject: Re: [cas-user] After a month, no tickets created in 4.2.2? > > Grepping through what I have, I can confirm that the tickets are being > removed, as I do have log statements like the following: > > catalina.out.2.gz:2016-07-08 22:19:31,948 INFO > [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - <1 expired tickets > found and removed.> > catalina.out.2.gz:2016-07-08 22:21:32,004 INFO > [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - <2 expired tickets > found and removed.> > > I'm using the default in memory registry, the default > tgt.maxTimeToLiveInSeconds + tgt.timeToKillInSeconds. > st.timeToKillInSeconds=60. > > Other pretty bare bones defaults: > > > > > > > > > > > alias="grantingTicketExpirationPolicy" /> > alias="serviceTicketExpirationPolicy" /> > > > > alias="authenticationPolicyFactory" /> > > Would you recommend perhaps a different ticket registry at this point > perhaps? I don't think I'm hitting the maximum amount of tickets that the > default ticket registry can hold by any means, since the maximum amount of > tickets I'm seeing expire is 20. Mostly it's between 0-2 tickets that are > expired, so I very much doubt the tickets being the memory bottleneck. > > --- > > Before I posted the above, I dug a little and noticed a strange ~200 > tickets being cleaned up a bit before the issue (an uptick in cleanup > tickets). Perhaps moving to a more robust ticket registry (not just in > memory) might actually help to mitigate then. > > Are service tickets also being cleaned up with the same default ticket > registry? What tickets should show up in that 'expired' count? TGT only, or > TGT + STs? > > For testing, I logged in to a test instance where the tgt expirey was set > to 10 seconds. 10 times: 15 expired tickets. I logged in 20 times. 29 > expired tickets. > > What other items are included in the expired count? > > On Wednesday, July 20, 2016 at 2:31:37 AM UTC-7, Misagh Moayyed wrote: >> >> Does your log show that tickets are cleaned up successfully? Is your >> expiration policy set up to allow the cleaner to expire and clean tickets >> successfully? >> >> Without logs, it’s just a guessing game. My bet is, somehow you’ve run >> out of memory. >> >> -- >> Misagh >> >> From: Jeffrey Wong >> Reply: Jeffrey Wong >> Date: July 19, 2016 at 3:10:05 PM >> To: CAS Community >> Subject: [cas-user] After a month, no tickets created in 4.2.2? >> >> Aft
Re: [cas-user] After a month, no tickets created in 4.2.2?
All expired tickets are removed by the cleaner, regardless of type. Come to think of it, do you use CAS for its proxy authentication features? That *might* have something to do with it, if you do. I personally don’t know if I’d recommend switching, because I don’t know what the problem is. Generally, switching to something more robust and distributed is a good idea, but if the problem is something else, it will simply repeat and your new registry will have done nothing to fix it. I would instead turn up logs, load test as much as possible and keep it running under test for some time. Observe. -- Misagh From: Jeffrey Wong Reply: Jeffrey Wong Date: July 20, 2016 at 3:27:17 PM To: CAS Community Cc: mmoay...@unicon.net Subject: Re: [cas-user] After a month, no tickets created in 4.2.2? Grepping through what I have, I can confirm that the tickets are being removed, as I do have log statements like the following: catalina.out.2.gz:2016-07-08 22:19:31,948 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - <1 expired tickets found and removed.> catalina.out.2.gz:2016-07-08 22:21:32,004 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - <2 expired tickets found and removed.> I'm using the default in memory registry, the default tgt.maxTimeToLiveInSeconds + tgt.timeToKillInSeconds. st.timeToKillInSeconds=60. Other pretty bare bones defaults: Would you recommend perhaps a different ticket registry at this point perhaps? I don't think I'm hitting the maximum amount of tickets that the default ticket registry can hold by any means, since the maximum amount of tickets I'm seeing expire is 20. Mostly it's between 0-2 tickets that are expired, so I very much doubt the tickets being the memory bottleneck. --- Before I posted the above, I dug a little and noticed a strange ~200 tickets being cleaned up a bit before the issue (an uptick in cleanup tickets). Perhaps moving to a more robust ticket registry (not just in memory) might actually help to mitigate then. Are service tickets also being cleaned up with the same default ticket registry? What tickets should show up in that 'expired' count? TGT only, or TGT + STs? For testing, I logged in to a test instance where the tgt expirey was set to 10 seconds. 10 times: 15 expired tickets. I logged in 20 times. 29 expired tickets. What other items are included in the expired count? On Wednesday, July 20, 2016 at 2:31:37 AM UTC-7, Misagh Moayyed wrote: Does your log show that tickets are cleaned up successfully? Is your expiration policy set up to allow the cleaner to expire and clean tickets successfully? Without logs, it’s just a guessing game. My bet is, somehow you’ve run out of memory. -- Misagh From: Jeffrey Wong Reply: Jeffrey Wong Date: July 19, 2016 at 3:10:05 PM To: CAS Community Subject: [cas-user] After a month, no tickets created in 4.2.2? After about a month of working perfectly on 4.2.2 deployed to tomcat7, running under java8, randomly the in-memory ticketing system would not create any more tickets. Restarting the tomcat instance fixed it, but I'm wondering why CAS would randomly break on me after working so well! Using a LDAP (AD) backed user base with a mysql backed attribute DB. We have pretty minimal traffic, so I'm not sure why I am seeing issues after such a small amount of time. Despite having an , no errors have been thrown at the time of issue. Unfortunately, I've only had the org.springframework.jdbc logger set to debug, and all others at info, so I have very minimal logging around the issue. I'm noticing both the ldap auth AND the jdbc handlers returning without issues (no errors). ...But no tickets? Here's a sample of the logs: 2016-07-19 16:28:16,399 INFO [org.jasig.cas.authentication.PolicyBasedAuthenticationManager] - 2016-07-19 16:28:16,400 DEBUG [org.springframework.jdbc.core.JdbcTemplate] - 2016-07-19 16:28:16,400 DEBUG [org.springframework.jdbc.core.JdbcTemplate] - 2016-07-19 16:28:16,400 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] - 2016-07-19 16:28:16,401 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] - 2016-07-19 16:28:19,015 INFO [org.jasig.cas.authentication.PolicyBasedAuthenticationManager] - 2016-07-19 16:28:19,015 DEBUG [org.sp
Re: [cas-user] After a month, no tickets created in 4.2.2?
Grepping through what I have, I can confirm that the tickets are being removed, as I do have log statements like the following: catalina.out.2.gz:2016-07-08 22:19:31,948 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - <1 expired tickets found and removed.> catalina.out.2.gz:2016-07-08 22:21:32,004 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - <2 expired tickets found and removed.> I'm using the default in memory registry, the default tgt.maxTimeToLiveInSeconds + tgt.timeToKillInSeconds. st.timeToKillInSeconds=60. Other pretty bare bones defaults: Would you recommend perhaps a different ticket registry at this point perhaps? I don't think I'm hitting the maximum amount of tickets that the default ticket registry can hold by any means, since the maximum amount of tickets I'm seeing expire is 20. Mostly it's between 0-2 tickets that are expired, so I very much doubt the tickets being the memory bottleneck. --- Before I posted the above, I dug a little and noticed a strange ~200 tickets being cleaned up a bit before the issue (an uptick in cleanup tickets). Perhaps moving to a more robust ticket registry (not just in memory) might actually help to mitigate then. Are service tickets also being cleaned up with the same default ticket registry? What tickets should show up in that 'expired' count? TGT only, or TGT + STs? For testing, I logged in to a test instance where the tgt expirey was set to 10 seconds. 10 times: 15 expired tickets. I logged in 20 times. 29 expired tickets. What other items are included in the expired count? On Wednesday, July 20, 2016 at 2:31:37 AM UTC-7, Misagh Moayyed wrote: > > Does your log show that tickets are cleaned up successfully? Is your > expiration policy set up to allow the cleaner to expire and clean tickets > successfully? > > Without logs, it’s just a guessing game. My bet is, somehow you’ve run out > of memory. > > -- > Misagh > > From: Jeffrey Wong > Reply: Jeffrey Wong > Date: July 19, 2016 at 3:10:05 PM > To: CAS Community > Subject: [cas-user] After a month, no tickets created in 4.2.2? > > After about a month of working perfectly on 4.2.2 deployed to tomcat7, > running under java8, randomly the in-memory ticketing system would not > create any more tickets. Restarting the tomcat instance fixed it, but I'm > wondering why CAS would randomly break on me after working so well! Using a > LDAP (AD) backed user base with a mysql backed attribute DB. We have pretty > minimal traffic, so I'm not sure why I am seeing issues after such a small > amount of time. > > Despite having an , no errors have been thrown at > the time of issue. > > Unfortunately, I've only had the org.springframework.jdbc logger set to > debug, and all others at info, so I have very minimal logging around the > issue. > > I'm noticing both the ldap auth AND the jdbc handlers returning without > issues (no errors). ...But no tickets? > > Here's a sample of the logs: > > 2016-07-19 16:28:16,399 INFO > [org.jasig.cas.authentication.PolicyBasedAuthenticationManager] - > > 2016-07-19 16:28:16,400 DEBUG [org.springframework.jdbc.core.JdbcTemplate] - > > 2016-07-19 16:28:16,400 DEBUG [org.springframework.jdbc.core.JdbcTemplate] - > ID,username,FirstName,LastName,Email FROM User WHERE UserName = ?]> > 2016-07-19 16:28:16,400 DEBUG > [org.springframework.jdbc.datasource.DataSourceUtils] - Connection from DataSource> > 2016-07-19 16:28:16,401 DEBUG > [org.springframework.jdbc.datasource.DataSourceUtils] - Connection to DataSource> > 2016-07-19 16:28:19,015 INFO > [org.jasig.cas.authentication.PolicyBasedAuthenticationManager] - > > 2016-07-19 16:28:19,015 DEBUG [org.springframework.jdbc.core.JdbcTemplate] - > > 2016-07-19 16:28:19,015 DEBUG [org.springframework.jdbc.core.JdbcTemplate] - > ID,username,FirstName,LastName,Email FROM User WHERE UserName = ?]> > 2016-07-19 16:28:19,015 DEBUG > [org.springframework.jdbc.datasource.DataSourceUtils] - Connection from DataSource> > 2016-07-19 16:28:19,017 DEBUG > [org.springframework.jdbc.datasource.DataSourceUtils] - Connection to DataSource> > > Immediately before this, I've seen tickets that are created (an audit log > is posted that a ticket granting ticket has been created and validated, and > all is good). There are no exceptions thrown between when the tickets were > able to be created and w
Re: [cas-user] After a month, no tickets created in 4.2.2?
Does your log show that tickets are cleaned up successfully? Is your expiration policy set up to allow the cleaner to expire and clean tickets successfully? Without logs, it’s just a guessing game. My bet is, somehow you’ve run out of memory. -- Misagh From: Jeffrey Wong Reply: Jeffrey Wong Date: July 19, 2016 at 3:10:05 PM To: CAS Community Subject: [cas-user] After a month, no tickets created in 4.2.2? After about a month of working perfectly on 4.2.2 deployed to tomcat7, running under java8, randomly the in-memory ticketing system would not create any more tickets. Restarting the tomcat instance fixed it, but I'm wondering why CAS would randomly break on me after working so well! Using a LDAP (AD) backed user base with a mysql backed attribute DB. We have pretty minimal traffic, so I'm not sure why I am seeing issues after such a small amount of time. Despite having an , no errors have been thrown at the time of issue. Unfortunately, I've only had the org.springframework.jdbc logger set to debug, and all others at info, so I have very minimal logging around the issue. I'm noticing both the ldap auth AND the jdbc handlers returning without issues (no errors). ...But no tickets? Here's a sample of the logs: 2016-07-19 16:28:16,399 INFO [org.jasig.cas.authentication.PolicyBasedAuthenticationManager] - 2016-07-19 16:28:16,400 DEBUG [org.springframework.jdbc.core.JdbcTemplate] - 2016-07-19 16:28:16,400 DEBUG [org.springframework.jdbc.core.JdbcTemplate] - 2016-07-19 16:28:16,400 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] - 2016-07-19 16:28:16,401 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] - 2016-07-19 16:28:19,015 INFO [org.jasig.cas.authentication.PolicyBasedAuthenticationManager] - 2016-07-19 16:28:19,015 DEBUG [org.springframework.jdbc.core.JdbcTemplate] - 2016-07-19 16:28:19,015 DEBUG [org.springframework.jdbc.core.JdbcTemplate] - 2016-07-19 16:28:19,015 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] - 2016-07-19 16:28:19,017 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] - Immediately before this, I've seen tickets that are created (an audit log is posted that a ticket granting ticket has been created and validated, and all is good). There are no exceptions thrown between when the tickets were able to be created and when there was this bottleneck. On the front end, after the logs say 'success' without a ticket created, they are redirected to the main cas login page. Reproducing this is also difficult as it will stop intermittently, without warning. What are the best ways to debug or resolve these sorts of issues? What could be causing this issue? Thanks in advance, Jeff -- You received this message because you are subscribed to the Google Groups "CAS Community" group. To unsubscribe from this group and stop receiving emails from it, send an email to cas-user+unsubscr...@apereo.org. To post to this group, send email to cas-user@apereo.org. Visit this group at https://groups.google.com/a/apereo.org/group/cas-user/. To view this discussion on the web visit https://groups.google.com/a/apereo.org/d/msgid/cas-user/23b96c4d-0997-4da1-9051-7ddf1560e8bb%40apereo.org. For more options, visit https://groups.google.com/a/apereo.org/d/optout. -- You received this message because you are subscribed to the Google Groups "CAS Community" group. To unsubscribe from this group and stop receiving emails from it, send an email to cas-user+unsubscr...@apereo.org. To post to this group, send email to cas-user@apereo.org. Visit this group at https://groups.google.com/a/apereo.org/group/cas-user/. To view this discussion on the web visit https://groups.google.com/a/apereo.org/d/msgid/cas-user/etPan.578f44ec.beaeb34.2d8f%40unicon.net. For more options, visit https://groups.google.com/a/apereo.org/d/optout.
