https://bz.apache.org/bugzilla/show_bug.cgi?id=62175
Bug ID: 62175 Summary: Infinite recursive session PersistentManager.swapIn loop with OOM Product: Tomcat 9 Version: 9.0.6 Hardware: PC OS: Windows NT Status: NEW Severity: normal Priority: P2 Component: Catalina Assignee: dev@tomcat.apache.org Reporter: holger.su...@bodo-peters.de Target Milestone: ----- Environment: Windows 8 NTFS Oracle JDK 1.8.0_152 Apache Tomcat 9.0.6 Apache OpenWebBeans 2.0.4 (OWB) Session ID ABCDE in filesystem storage and containing an expired session. TL;DR: Expired session is deserialized over and over again because OWB calls Request.getSession(false) in sessionDestroyed and ContainerBackgroundProcessor is unable to delete the session file while it is deserialized. While Tomcat is starting up, I direct the Browser to request my application with session ABCDE which had been swapped out but is already expired. Tomcat calls StandardSession.isValid() -> StandardSession.expire() -> each registered HttpSessionListener.destroy() Now OWB having a HttpSessionListener registered is calling RequestFacade.getSession(false) which expands to -> PersistentManager.findSession() -> PersistentManager.swapIn() -> StandardSession.isValid() ... repeat until OutOfMemory An important side condition is that FileStore.remove() fails for unknown reason. File.delete just returns false which is not very expressive. Maybe the session file is beeing read / locked accidentally the same time while the ContainerBackgroundProcessor tries to delete it. This could be caused by the session beeing read over and over again by that recursive loop where the most expansive operation is deserialization of the session, giving close to zero time for a successful delete. [This is one reason why I better like Linux: deleting a file while it is read simply deletes the file.] BTW: When the HTTP-Thread is blocked by a breakpoint, ContainerBackgroundProcessor succeeds in deleting the file and stops the recursion. Here are the stacktraces of HTTP thread and ContainerBackgroundProcessor: "http-nio-8080-exec-2" #66 daemon prio=5 os_prio=0 tid=0x0000000021e9b000 nid=0xf94 at breakpoint[0x000000002c40a000] java.lang.Thread.State: RUNNABLE at my.web.common.application.listener.ApplicationListener.sessionDestroyed(ApplicationListener.java:112) at org.apache.catalina.session.StandardSession.expire(StandardSession.java:801) - locked <0x00000000d709d800> (a org.apache.catalina.session.StandardSession) at org.apache.catalina.session.StandardSession.isValid(StandardSession.java:659) at org.apache.catalina.session.PersistentManagerBase.swapIn(PersistentManagerBase.java:736) - locked <0x00000000fa5be668> (a java.lang.Object) at org.apache.catalina.session.PersistentManagerBase.findSession(PersistentManagerBase.java:488) at org.apache.catalina.connector.Request.doGetSession(Request.java:2909) at org.apache.catalina.connector.Request.getSession(Request.java:2374) at org.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:908) at org.apache.webbeans.web.context.WebContextsService.destroySessionContext(WebContextsService.java:553) at org.apache.webbeans.web.context.WebContextsService.endContext(WebContextsService.java:228) at org.apache.webbeans.servlet.WebBeansConfigurationListener.sessionDestroyed(WebBeansConfigurationListener.java:197) at org.apache.catalina.session.StandardSession.expire(StandardSession.java:801) - locked <0x00000000d61c8d48> (a org.apache.catalina.session.StandardSession) at org.apache.catalina.session.StandardSession.isValid(StandardSession.java:659) at org.apache.catalina.session.PersistentManagerBase.swapIn(PersistentManagerBase.java:736) - locked <0x00000000fa5be668> (a java.lang.Object) at org.apache.catalina.session.PersistentManagerBase.findSession(PersistentManagerBase.java:488) at org.apache.catalina.connector.Request.doGetSession(Request.java:2909) at org.apache.catalina.connector.Request.getSession(Request.java:2374) at org.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:908) at org.apache.webbeans.web.context.WebContextsService.destroySessionContext(WebContextsService.java:553) at org.apache.webbeans.web.context.WebContextsService.endContext(WebContextsService.java:228) at org.apache.webbeans.servlet.WebBeansConfigurationListener.sessionDestroyed(WebBeansConfigurationListener.java:197) at org.apache.catalina.session.StandardSession.expire(StandardSession.java:801) - locked <0x00000000fbb10008> (a org.apache.catalina.session.StandardSession) at org.apache.catalina.session.StandardSession.isValid(StandardSession.java:659) at org.apache.catalina.session.PersistentManagerBase.swapIn(PersistentManagerBase.java:736) - locked <0x00000000fa5be668> (a java.lang.Object) at org.apache.catalina.session.PersistentManagerBase.findSession(PersistentManagerBase.java:488) at org.apache.catalina.connector.Request.doGetSession(Request.java:2909) at org.apache.catalina.connector.Request.getSession(Request.java:2374) at org.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:908) at org.apache.webbeans.web.context.WebContextsService.destroySessionContext(WebContextsService.java:553) at org.apache.webbeans.web.context.WebContextsService.endContext(WebContextsService.java:228) at org.apache.webbeans.servlet.WebBeansConfigurationListener.sessionDestroyed(WebBeansConfigurationListener.java:197) at org.apache.catalina.session.StandardSession.expire(StandardSession.java:801) - locked <0x00000000fbb100d8> (a org.apache.catalina.session.StandardSession) at org.apache.catalina.session.StandardSession.isValid(StandardSession.java:659) at org.apache.catalina.session.PersistentManagerBase.swapIn(PersistentManagerBase.java:736) - locked <0x00000000fa5be668> (a java.lang.Object) at org.apache.catalina.session.PersistentManagerBase.findSession(PersistentManagerBase.java:488) at org.apache.catalina.connector.Request.doGetSession(Request.java:2909) at org.apache.catalina.connector.Request.getSession(Request.java:2374) at org.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:908) at org.apache.webbeans.web.context.WebContextsService.destroySessionContext(WebContextsService.java:553) at org.apache.webbeans.web.context.WebContextsService.endContext(WebContextsService.java:228) at org.apache.webbeans.servlet.WebBeansConfigurationListener.sessionDestroyed(WebBeansConfigurationListener.java:197) at org.apache.catalina.session.StandardSession.expire(StandardSession.java:801) - locked <0x00000000fbb10150> (a org.apache.catalina.session.StandardSession) at org.apache.catalina.session.StandardSession.isValid(StandardSession.java:659) at org.apache.catalina.session.PersistentManagerBase.swapIn(PersistentManagerBase.java:736) - locked <0x00000000fa5be668> (a java.lang.Object) at org.apache.catalina.session.PersistentManagerBase.findSession(PersistentManagerBase.java:488) at org.apache.catalina.connector.Request.doGetSession(Request.java:2909) at org.apache.catalina.connector.Request.getSession(Request.java:2374) at org.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:908) at org.apache.webbeans.web.context.WebContextsService.destroySessionContext(WebContextsService.java:553) at org.apache.webbeans.web.context.WebContextsService.endContext(WebContextsService.java:228) at org.apache.webbeans.servlet.WebBeansConfigurationListener.sessionDestroyed(WebBeansConfigurationListener.java:197) at org.apache.catalina.session.StandardSession.expire(StandardSession.java:801) - locked <0x00000000fbb101c8> (a org.apache.catalina.session.StandardSession) at org.apache.catalina.session.StandardSession.isValid(StandardSession.java:659) at org.apache.catalina.session.PersistentManagerBase.swapIn(PersistentManagerBase.java:736) - locked <0x00000000fa5be668> (a java.lang.Object) at org.apache.catalina.session.PersistentManagerBase.findSession(PersistentManagerBase.java:488) at org.apache.catalina.connector.Request.doGetSession(Request.java:2909) at org.apache.catalina.connector.Request.getSession(Request.java:2374) at org.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:908) at org.apache.webbeans.web.context.WebContextsService.destroySessionContext(WebContextsService.java:553) at org.apache.webbeans.web.context.WebContextsService.endContext(WebContextsService.java:228) at org.apache.webbeans.servlet.WebBeansConfigurationListener.sessionDestroyed(WebBeansConfigurationListener.java:197) at org.apache.catalina.session.StandardSession.expire(StandardSession.java:801) - locked <0x00000000fbb10240> (a org.apache.catalina.session.StandardSession) at org.apache.catalina.session.StandardSession.isValid(StandardSession.java:659) at org.apache.catalina.session.PersistentManagerBase.swapIn(PersistentManagerBase.java:736) - locked <0x00000000fa5be668> (a java.lang.Object) at org.apache.catalina.session.PersistentManagerBase.findSession(PersistentManagerBase.java:488) at org.apache.catalina.connector.Request.doGetSession(Request.java:2909) at org.apache.catalina.connector.Request.getSession(Request.java:2374) at org.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:908) at org.apache.webbeans.web.context.WebContextsService.destroySessionContext(WebContextsService.java:553) at org.apache.webbeans.web.context.WebContextsService.endContext(WebContextsService.java:228) at org.apache.webbeans.servlet.WebBeansConfigurationListener.sessionDestroyed(WebBeansConfigurationListener.java:197) at org.apache.catalina.session.StandardSession.expire(StandardSession.java:801) - locked <0x00000000fbb102b8> (a org.apache.catalina.session.StandardSession) at org.apache.catalina.session.StandardSession.isValid(StandardSession.java:659) at org.apache.catalina.session.PersistentManagerBase.swapIn(PersistentManagerBase.java:736) - locked <0x00000000fa5be668> (a java.lang.Object) at org.apache.catalina.session.PersistentManagerBase.findSession(PersistentManagerBase.java:488) at org.apache.catalina.connector.Request.doGetSession(Request.java:2909) at org.apache.catalina.connector.Request.getSession(Request.java:2374) at org.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:908) at org.apache.webbeans.web.context.WebContextsService.destroySessionContext(WebContextsService.java:553) at org.apache.webbeans.web.context.WebContextsService.endContext(WebContextsService.java:228) at org.apache.webbeans.servlet.WebBeansConfigurationListener.sessionDestroyed(WebBeansConfigurationListener.java:197) at org.apache.catalina.session.StandardSession.expire(StandardSession.java:801) - locked <0x00000000fbb10330> (a org.apache.catalina.session.StandardSession) at org.apache.catalina.session.StandardSession.isValid(StandardSession.java:659) at org.apache.catalina.session.PersistentManagerBase.swapIn(PersistentManagerBase.java:736) - locked <0x00000000fa5be668> (a java.lang.Object) at org.apache.catalina.session.PersistentManagerBase.findSession(PersistentManagerBase.java:488) at org.apache.catalina.connector.Request.doGetSession(Request.java:2909) at org.apache.catalina.connector.Request.getSession(Request.java:2374) at org.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:908) at org.apache.webbeans.web.context.WebContextsService.destroySessionContext(WebContextsService.java:553) at org.apache.webbeans.web.context.WebContextsService.endContext(WebContextsService.java:228) at org.apache.webbeans.servlet.WebBeansConfigurationListener.sessionDestroyed(WebBeansConfigurationListener.java:197) at org.apache.catalina.session.StandardSession.expire(StandardSession.java:801) - locked <0x00000000fbb103a8> (a org.apache.catalina.session.StandardSession) at org.apache.catalina.session.StandardSession.isValid(StandardSession.java:659) at org.apache.catalina.session.PersistentManagerBase.swapIn(PersistentManagerBase.java:736) - locked <0x00000000fa5be668> (a java.lang.Object) at org.apache.catalina.session.PersistentManagerBase.findSession(PersistentManagerBase.java:488) at org.apache.catalina.connector.Request.doGetSession(Request.java:2909) at org.apache.catalina.connector.Request.getSession(Request.java:2374) at org.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:908) at org.apache.webbeans.web.context.WebContextsService.destroySessionContext(WebContextsService.java:553) at org.apache.webbeans.web.context.WebContextsService.endContext(WebContextsService.java:228) at org.apache.webbeans.servlet.WebBeansConfigurationListener.sessionDestroyed(WebBeansConfigurationListener.java:197) at org.apache.catalina.session.StandardSession.expire(StandardSession.java:801) - locked <0x00000000fbb10420> (a org.apache.catalina.session.StandardSession) at org.apache.catalina.session.StandardSession.isValid(StandardSession.java:659) at org.apache.catalina.session.PersistentManagerBase.swapIn(PersistentManagerBase.java:736) - locked <0x00000000fa5be668> (a java.lang.Object) at org.apache.catalina.session.PersistentManagerBase.findSession(PersistentManagerBase.java:488) at org.apache.catalina.connector.Request.doGetSession(Request.java:2909) at org.apache.catalina.connector.Request.getSession(Request.java:2374) at org.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:908) at org.apache.webbeans.web.context.WebContextsService.destroySessionContext(WebContextsService.java:553) at org.apache.webbeans.web.context.WebContextsService.endContext(WebContextsService.java:228) at org.apache.webbeans.servlet.WebBeansConfigurationListener.sessionDestroyed(WebBeansConfigurationListener.java:197) at org.apache.catalina.session.StandardSession.expire(StandardSession.java:801) - locked <0x00000000fbb10498> (a org.apache.catalina.session.StandardSession) at org.apache.catalina.session.StandardSession.isValid(StandardSession.java:659) at org.apache.catalina.session.PersistentManagerBase.swapIn(PersistentManagerBase.java:736) - locked <0x00000000fa5be668> (a java.lang.Object) at org.apache.catalina.session.PersistentManagerBase.findSession(PersistentManagerBase.java:488) at org.apache.catalina.connector.Request.doGetSession(Request.java:2909) at org.apache.catalina.connector.Request.getSession(Request.java:2374) at org.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:908) at org.apache.webbeans.web.context.WebContextsService.destroySessionContext(WebContextsService.java:553) at org.apache.webbeans.web.context.WebContextsService.endContext(WebContextsService.java:228) at org.apache.webbeans.servlet.WebBeansConfigurationListener.sessionDestroyed(WebBeansConfigurationListener.java:197) at org.apache.catalina.session.StandardSession.expire(StandardSession.java:801) - locked <0x00000000fbb10510> (a org.apache.catalina.session.StandardSession) at org.apache.catalina.session.StandardSession.isValid(StandardSession.java:659) at org.apache.catalina.session.PersistentManagerBase.swapIn(PersistentManagerBase.java:736) - locked <0x00000000fa5be668> (a java.lang.Object) at org.apache.catalina.session.PersistentManagerBase.findSession(PersistentManagerBase.java:488) at org.apache.catalina.connector.Request.doGetSession(Request.java:2909) at org.apache.catalina.connector.Request.getSessionInternal(Request.java:2635) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:464) at org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java:540) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:651) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:651) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:407) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:754) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1376) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) - locked <0x00000000f898c128> (a org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) java.io.IOException: Unable to delete file [....ABCDE.session] which is no longer required at org.apache.catalina.session.FileStore.remove(FileStore.java:269) at org.apache.catalina.session.StoreBase.processExpires(StoreBase.java:189) at org.apache.catalina.session.PersistentManagerBase.processExpires(PersistentManagerBase.java:433) at org.apache.catalina.session.ManagerBase.backgroundProcess(ManagerBase.java:558) at org.apache.catalina.core.StandardContext.backgroundProcess(StandardContext.java:5406) at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1396) at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1400) at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1400) at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1368) at java.lang.Thread.run(Thread.java:748) -- You are receiving this mail because: You are the assignee for the bug. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org