-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Stephen,
On 6/3/16 8:47 PM, Stephen Bhadran wrote: > *Linux version:* 2.6.32-279.5.2.el6.x86_64 ( > mockbu...@c6b10.bsys.dev.centos.org) (gcc version 4.4.6 20120305 > (Red Hat 4.4.6-4) (GCC) ) *Java Version: *1.7.0_55 *Tomcat > Version: *7.0.40 *Session Timeout: *30 minutes > > Hi, Two of our servers reached max heap and we had to restart to > resolve the issue. > > In troubleshooting, we noticed Http Sessions had stopped expiring > 05/12th and reached 25K over a period of 11 days. I noticed the > same behavior in both servers, the session expiration seem to have > stopped on both of them around the same time. I'm saying this > because from looking at the monitoring graphs (NewRelic) I notice > the http sessions starting to climb on both servers around the same > time frame. > > I am thinking, the issue could be two folds: #1 - The session > expiration stopped working -OR #2 - The sessions themselves were > created with TTLs way into the future > > I suspected #1. I looked that tomcat 7.0.40 source code, extracted > all session/manager error messages and searched for them in the > logs and didn't find any hits. I was hoping to find something about > "Session expiration stopped working", but didn't find any. > > Can you please advise how to troubleshoot this issue? I'd very > much appreciate the help. Were there any errors around the 12th? I've found that in rare cases, the server hits an OOME while the BackgroundProcessor thread is running, which kills the BackgroundProcessor. Guess which thread is responsible for triggering the session-cleaner? If the BackgroundProcessor died, it should have dumped an error to a log file. As I said, it's pretty rare, but it does happen and then it's only a matter of time before your sessions bust your heap. You can manually-trigger the session-cleaner via JMX of you absolutely have to, but I'd personally prefer to bounce a Tomcat instance in that case. - -chris -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJXUid5AAoJEBzwKT+lPKRYJfYQAKA2ditjB7B2vLbm0po83Tr0 YD2EjqLBrF4O/3Wu6enIDRAQEIGgXmTr2bgPlMvu+6aoDopiCSKltRxWQFlTMyIP 8ura/0YlS5sX3yGPPKN5nrZnOz2dvrgPncHLNVfuM9BEm+8QZPeKNZVDhMH53m1v 4YcLCM5IjswAS4FrVeG9A0gcZplBd53WUuypnRYQMUcS1sACdItoZ3TLAwb/WBg/ T9yoyftEXJVtP6lovkaWka3yw3v382P6HmSV6/TtCWjogB/tBdCsEPmGEQ0j7k8y 9dLmmTBawDdckaxiGf1M1M/5R+f7XqU4e02Dnhk9v4umysmu5EH+VQFO4kfi5qX+ DvaoS13lluyN1LGMyuwHkYFHXxlgZhR/XAW9Cn/0D/NDCQks6lHG9Sby+T/kvLSO YNSBV78SX9wsJ/fH9MxrcTarxyUYsxnHdNkcarkmcrQIkhda/vBXQobXpYVL9poR HmxNGNOujCTojqPJzlpgAaUVUq6pEUcN3u1vkCqXNQGVo983PHz2JhfEBGYihKBk icbuFxfIy2Dv6KwuPPMV9Y7nVRNVBrVEpieZuAc7Sj9/U3kUgPUznUSbRFUCnGd1 YI/kq9rn/qj/aqhaJdMF0Ci4+ieJMorKNUE1PWOjbnya8XoyBxXfPd5iV6n7elEw V6qNyO6n46isrAiVEu5c =MyIX -----END PGP SIGNATURE----- --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org