On Sat, Sep 28, 2019 at 9:05 PM Mark Thomas <ma...@apache.org> wrote:
> On 27/09/2019 22:39, Chen Levy wrote: > > -----Original Message----- > > From: Mark Thomas <ma...@apache.org> > > Sent: Friday, September 27, 2019 15:34 > > To: users@tomcat.apache.org > > Subject: Re: Tomcat 9.0.24/9.0.26 suspected memory leak > > > > On 27/09/2019 16:34, Chen Levy wrote: > >> On 26/09/2019 18:22, Chen Levy wrote: > > > > <snip/> > > > >>> The HashMap referenced in the report appears to be "waitingProcessors" > inside AbstractProtocol which contain 262K entries. > >> > >> OK. Those are asynchronous Servlets that are still in async mode. > > > > <snip/> > > > >> * I do not employ async servlets in my application > > > > OK. Do you use WebSocket? There is a code path to add Processors to the > waitingProcessors Map for WebSocket as well. > > > > Mark > > > > > > No, no WebSocket either; just plain old Servlets, Filters and the > occasional JSP > > OK. That narrows down where/how this might be happening. > > What about if you disable HTTP/2. Do you still see the issue then? > I added debug code in AbstractProtocol.ConnectionHandler.release(SocketWrapperBase<S>) to check if the processor considered was present in the waitingProcessors map. The result is the following: TEST-javax.servlet.http.TestHttpServletResponseSendError.NIO.txt:CHECK PROCESSOR FAILED org.apache.coyote.http11.Http11Processor@77b16580 TEST-javax.servlet.http.TestHttpServletResponseSendError.NIO.txt:CHECK PROCESSOR FAILED org.apache.coyote.http11.Http11Processor@1d902704 TEST-javax.servlet.http.TestHttpServletResponseSendError.NIO.txt:CHECK PROCESSOR FAILED org.apache.coyote.http11.Http11Processor@610c4fc8 TEST-javax.servlet.http.TestHttpServletResponseSendError.NIO.txt:CHECK PROCESSOR FAILED org.apache.coyote.http11.Http11Processor@1a3a3cb6 TEST-javax.servlet.http.TestHttpServletResponseSendError.NIO.txt:CHECK PROCESSOR FAILED org.apache.coyote.http11.Http11Processor@336f552d TEST-javax.servlet.http.TestHttpServletResponseSendError.NIO.txt:CHECK PROCESSOR FAILED org.apache.coyote.http11.Http11Processor@3cd94f25 TEST-javax.servlet.http.TestHttpServletResponseSendError.NIO.txt:CHECK PROCESSOR FAILED org.apache.coyote.http11.Http11Processor@66e24762 TEST-javax.servlet.http.TestHttpServletResponseSendError.NIO.txt:CHECK PROCESSOR FAILED org.apache.coyote.http11.Http11Processor@7c7a1c3c TEST-org.apache.coyote.http11.TestHttp11Processor.NIO.txt:CHECK PROCESSOR FAILED org.apache.coyote.http11.Http11Processor@55a44822 TEST-org.apache.coyote.http11.upgrade.TestUpgradeInternalHandler.NIO.txt:CHECK PROCESSOR FAILED org.apache.coyote.http11.upgrade.UpgradeProcessorInternal@6e55ff60 TEST-org.apache.coyote.http11.upgrade.TestUpgrade.NIO.txt:CHECK PROCESSOR FAILED org.apache.coyote.http11.upgrade.UpgradeProcessorExternal@37d98b7f TEST-org.apache.tomcat.websocket.server.TestShutdown.NIO.txt:CHECK PROCESSOR FAILED org.apache.coyote.http11.upgrade.UpgradeProcessorInternal@6be9bd85 TEST-org.apache.tomcat.websocket.TestWsRemoteEndpoint.NIO.txt:CHECK PROCESSOR FAILED org.apache.coyote.http11.upgrade.UpgradeProcessorInternal@3bd4e02f TEST-org.apache.tomcat.websocket.TestWsRemoteEndpoint.NIO.txt:CHECK PROCESSOR FAILED org.apache.coyote.http11.upgrade.UpgradeProcessorInternal@4bb23a77 TEST-org.apache.tomcat.websocket.TestWsRemoteEndpoint.NIO.txt:CHECK PROCESSOR FAILED org.apache.coyote.http11.upgrade.UpgradeProcessorInternal@32e20d65 TEST-org.apache.tomcat.websocket.TestWsRemoteEndpoint.NIO.txt:CHECK PROCESSOR FAILED org.apache.coyote.http11.upgrade.UpgradeProcessorInternal@16abf52f All instances of not removed processors are either from async or upgraded processors (the internal kind), as expected. I have verified the processor instances above are never removed so it might be more robust to simply call proto.removeWaitingProcessor(processor); in AbstractProtocol.ConnectionHandler.release(SocketWrapperBase<S>) (after all the socket is closed and done after that point). There could be a more fine grained solution of course. However, this does not match the leak scenario described by the user, this doesn't happen without async or websockets being used. Rémy
