Hi, We have experienced similar behavior. I have posted previously about this issue so forgive me if I repeat myself here.
Some background first based on our recent experiences. We noticed an increase in CPU usage and disk I/O after we migrated from Tomcat 7.0.55 to Tomcat 9.0.19. What changed? Tomcat has a (Context, Engine, Host, Loader) maintenance thread that performs various tasks (backgroundProcess()) and runs every 10 seconds by default (or configurable by Engine.backgroundProcessorDelay). This maintenance thread is not new and runs in Tomcat 7/8/9; it is started for each web application after it is successfully initialized. Some of those tasks include deleting expired sessions and closing any open files handles (such as the web application jars). The Tomcat 7 WebappClassLoader.java implementation conditionally closes jars. It will only close the set of jars associated with a web application if they were not accessed within the last 90 second period called the jarOpenInterval. The purpose of the jarOpenInterval appears to be to avoid processing delays when loading classes as the WebappClassLoader is singled threaded. In Tomcat 7.0.84 or later, that period was made configurable by the jarOpenInterval property (the WebappClassLoader was refactored into WebappClassLoaderBase). The jarOpenInterval feature was removed in Tomcat 8.5 (probably because of the addition of the ParallelWebappClassLoader); therefore, in Tomcat 8.5 and later, backgroundProcess() unconditionally unloads the jars (StandardRoot.gc()). When the application requires a class that is not currently in the class loader cache, the class path is searched and then it is cached until its needed. After a period of time, we expect all the relevant classes to be loaded into the cache and the jars to close without the need to open again. However, there are cases where the jars are being repeatedly searched for the same class or file (why search again, if it was found, it should be cached?). This behavior happens regardless of what Tomcat version our web applications are deployed; however, when deployed in Tomcat 7, it is not apparent since the jars are read (searched) often enough in an active environment (within 90 seconds) and not closed. Ok - what about DocumentBuilderFactory.newInstance()? The jar reload issue appears to be related to various class and resource search algorithms, such as ServiceLoader (https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html). The following usages contribute to this behavior (there are others) XML parsing DocumentBuilderFactory.newInstance() XML transformations TransformerFactory.newInstance() XPathFactory.newInstance() How to resolve? We are taking multiple approaches. Here are a few: 1. We implement wrapper classes around any applicable factory to get the requested implementation once and then subsequently reuse; reuse could mean using a singleton instance or just caching the factory implementation class that was found and creating new instances based on that class (again avoid the search). 2. Avoid the ServiceLoader search by setting the applicable factory system property on the JVM when starting Tomcat instances : like: -Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl Here is my previous post: http://mail-archives.apache.org/mod_mbox/tomcat-users/201910.mbox/%3CDM5PR0102MB336678F059DBD91B83ED8BD580950%40DM5PR0102MB3366.prod.exchangelabs.com%3E -----Original Message----- From: Niall Fitzpatrick <niallfitzpatr...@live.co.uk> Sent: Tuesday, January 14, 2020 9:41 AM To: Tomcat Users List <users@tomcat.apache.org> Subject: Re: Class loader takes long time after server inactivity - Tomcat Version 9.0.10 > Hi Folks, > > > > I have a web application that, after a period of inactivity (1-2hours), will > stall for 3 - 35 seconds upon the first new session. All subsequent sessions > will not experience this delay unless another period of inactivity occurs. > > > > I've found that the delay occurs when loading jars that live in the > WEB-INF/lib folder. > > > > The issue doesn't occur when the server is first started. Only after a period > of 1-2 hours approx. will the next session experience the delay when loading > a jar. This leads me to believe that when the server is started 'knowledge' > of the jars might be cached, however, after a period of inactivity this cache > may expire. The result of this is that the next session after this period > causes the lib folder to be searched again which seems to take the 5-35 > seconds. > > > > My question: Is this theory above correct, and if so, is there a way to > configure Tomcat such that this cache doesn't expire so that it doesn't need > to be re-populated when a new session begins? It depends. Which resources are slow to load? Mark Hi Mark, Doesn't seem to matter which Jar. At first it happened when loading a class from my own custom Jar. I removed this code to determine if it was the cause. However, then I found the same problem further downstream. When calling DocumentBuilderFactory.newInstance() which lives in xml-apis-1.3.04 The delay of 5-35 seconds occurred here instead. Niall --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org CONFIDENTIALITY NOTICE This message and any included attachments are from Cerner Corporation and are intended only for the addressee. The information contained in this message is confidential and may constitute inside or non-public information under international, federal, or state securities laws. Unauthorized forwarding, printing, copying, distribution, or use of such information is strictly prohibited and may be unlawful. If you are not the addressee, please promptly delete this message and notify the sender of the delivery error by e-mail or you may call Cerner's corporate offices in Kansas City, Missouri, U.S.A at (+1) (816)221-1024. --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
