Async servlet non-blocking IO and request recycling
Hi, I have a question regarding async servlets and request recycling. In particular, I'm interested in request recycling by tomcat when an IO error occurs. In my use case, I'm using non-blocking reads with a ReadListener but blocking writes (no WriteListener). The main question can be summarized as the following: Does tomcat always call the AsyncListener's onComplete method before recycling the request and response objects? I assumed that this must be true, but I have discovered that it is not so. My scenario very much resembles the one described in the following thread from 2016 except that the IO exception occurs on a container thread: http://mail-archives.apache.org/mod_mbox/tomcat-users/201611.mbox/%3c0d28a20f-fa3d-9e58-b5d7-82950ca51...@apache.org%3e Also, my scenario is extremely similar to the scenarios described in the following bug reports: https://bz.apache.org/bugzilla/show_bug.cgi?id=59219 https://bz.apache.org/bugzilla/show_bug.cgi?id=59220 Those bugs were mainly fixed by modifying the service method of the CoyoteAdapter class, but in my scenario, it looks like it's the asyncDispatch method that is being hit instead. Description of the scenario In a servlet, start async processing and register a read listener for non-blocking IO processing. In the read listener, perform blocking writes on the response when the onAllDataRead callback is called. Importantly, catch IOExceptions and don't let them bubble up. Also start a non-container thread that will access the request and/or response object some time in the future. This processing will be prevented if the AsyncListener's onComplete method is called. We can assume that this is properly synchronized. Have a client access this servlet, but immediately disconnect without waiting for the response. This will cause the write operations in the ReadListener's onAllDataRead method to throw IOExceptions, but as mentioned previously this exception is caught and silenced by the servlet. The AsyncContext complete method is also NOT called by the servlet at any time. Tomcat then processes the IO error and recycles the request and response objects. Some time later, the non-container thread attempts to access the request and/or response objects and gets hit with a non-checked exception because they have been recycled. This behavior has been observed in tomcat 8.0.53 and 9.0.24. I have not tested it on 9.0.30 yet. There are a few changes related to async processing since 9.0.24 but none of them seem related to that scenario. For the record, here they are (from the changelog): 63682: Fix a potential hang when using the asynchronous Servlet API to write the response body and the stream and/or connection window reaches 0 bytes in size. (markt) 63816 and 63817: Correctly handle I/O errors after asynchronous processing has been started but before the container thread that started asynchronous processing has completed processing the current request/response. (markt) Ensure that ServletRequest.isAsyncStarted() returns false once AsyncContext.complete() or AsyncContext.dispatch() has been called during AsyncListener.onTimeout() or AsyncListener.onError(). (markt) 63931: Improve timeout handling for asyncIO to ensure that blocking operations see a SocketTimeoutException if one occurs. (remm/markt) Please let me know if my understanding of the servlet API is wrong regarding asynchronous operation and non-blocking IO processing! I can adjust my servlet to do the appropriate cleanup when an IOException is caught, but it seems so much hackier and uglier than just processing an onComplete or onError callback. Regards, François - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Class loader takes long time after server inactivity - Tomcat Version 9.0.10
On 14/01/2020 13:42, Niall Fitzpatrick wrote: > Hi Folks, > > > > I have a web application that, after a period of inactivity (1-2hours), will > stall for 3 – 35 seconds upon the first new session. All subsequent sessions > will not experience this delay unless another period of inactivity occurs. > > > > I’ve found that the delay occurs when loading jars that live in the > WEB-INF/lib folder. > > > > The issue doesn’t occur when the server is first started. Only after a period > of 1-2 hours approx. will the next session experience the delay when loading > a jar. This leads me to believe that when the server is started ‘knowledge’ > of the jars might be cached, however, after a period of inactivity this cache > may expire. The result of this is that the next session after this period > causes the lib folder to be searched again which seems to take the 5-35 > seconds. > > > > My question: Is this theory above correct, and if so, is there a way to > configure Tomcat such that this cache doesn’t expire so that it doesn’t need > to be re-populated when a new session begins? It depends. Which resources are slow to load? Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Class loader takes long time after server inactivity - Tomcat Version 9.0.10
> Hi Folks, > > > > I have a web application that, after a period of inactivity (1-2hours), will > stall for 3 – 35 seconds upon the first new session. All subsequent sessions > will not experience this delay unless another period of inactivity occurs. > > > > I’ve found that the delay occurs when loading jars that live in the > WEB-INF/lib folder. > > > > The issue doesn’t occur when the server is first started. Only after a period > of 1-2 hours approx. will the next session experience the delay when loading > a jar. This leads me to believe that when the server is started ‘knowledge’ > of the jars might be cached, however, after a period of inactivity this cache > may expire. The result of this is that the next session after this period > causes the lib folder to be searched again which seems to take the 5-35 > seconds. > > > > My question: Is this theory above correct, and if so, is there a way to > configure Tomcat such that this cache doesn’t expire so that it doesn’t need > to be re-populated when a new session begins? It depends. Which resources are slow to load? Mark Hi Mark, Doesn't seem to matter which Jar. At first it happened when loading a class from my own custom Jar. I removed this code to determine if it was the cause. However, then I found the same problem further downstream. When calling DocumentBuilderFactory.newInstance() which lives in xml-apis-1.3.04 The delay of 5-35 seconds occurred here instead. Niall - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Class loader takes long time after server inactivity - Tomcat Version 9.0.10
Hi, We have experienced similar behavior. I have posted previously about this issue so forgive me if I repeat myself here. Some background first based on our recent experiences. We noticed an increase in CPU usage and disk I/O after we migrated from Tomcat 7.0.55 to Tomcat 9.0.19. What changed? Tomcat has a (Context, Engine, Host, Loader) maintenance thread that performs various tasks (backgroundProcess()) and runs every 10 seconds by default (or configurable by Engine.backgroundProcessorDelay). This maintenance thread is not new and runs in Tomcat 7/8/9; it is started for each web application after it is successfully initialized. Some of those tasks include deleting expired sessions and closing any open files handles (such as the web application jars). The Tomcat 7 WebappClassLoader.java implementation conditionally closes jars. It will only close the set of jars associated with a web application if they were not accessed within the last 90 second period called the jarOpenInterval. The purpose of the jarOpenInterval appears to be to avoid processing delays when loading classes as the WebappClassLoader is singled threaded. In Tomcat 7.0.84 or later, that period was made configurable by the jarOpenInterval property (the WebappClassLoader was refactored into WebappClassLoaderBase). The jarOpenInterval feature was removed in Tomcat 8.5 (probably because of the addition of the ParallelWebappClassLoader); therefore, in Tomcat 8.5 and later, backgroundProcess() unconditionally unloads the jars (StandardRoot.gc()). When the application requires a class that is not currently in the class loader cache, the class path is searched and then it is cached until its needed. After a period of time, we expect all the relevant classes to be loaded into the cache and the jars to close without the need to open again. However, there are cases where the jars are being repeatedly searched for the same class or file (why search again, if it was found, it should be cached?). This behavior happens regardless of what Tomcat version our web applications are deployed; however, when deployed in Tomcat 7, it is not apparent since the jars are read (searched) often enough in an active environment (within 90 seconds) and not closed. Ok - what about DocumentBuilderFactory.newInstance()? The jar reload issue appears to be related to various class and resource search algorithms, such as ServiceLoader (https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html). The following usages contribute to this behavior (there are others) XML parsing DocumentBuilderFactory.newInstance() XML transformations TransformerFactory.newInstance() XPathFactory.newInstance() How to resolve? We are taking multiple approaches. Here are a few: 1. We implement wrapper classes around any applicable factory to get the requested implementation once and then subsequently reuse; reuse could mean using a singleton instance or just caching the factory implementation class that was found and creating new instances based on that class (again avoid the search). 2. Avoid the ServiceLoader search by setting the applicable factory system property on the JVM when starting Tomcat instances : like: -Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl Here is my previous post: http://mail-archives.apache.org/mod_mbox/tomcat-users/201910.mbox/%3CDM5PR0102MB336678F059DBD91B83ED8BD580950%40DM5PR0102MB3366.prod.exchangelabs.com%3E -Original Message- From: Niall Fitzpatrick Sent: Tuesday, January 14, 2020 9:41 AM To: Tomcat Users List Subject: Re: Class loader takes long time after server inactivity - Tomcat Version 9.0.10 > Hi Folks, > > > > I have a web application that, after a period of inactivity (1-2hours), will > stall for 3 - 35 seconds upon the first new session. All subsequent sessions > will not experience this delay unless another period of inactivity occurs. > > > > I've found that the delay occurs when loading jars that live in the > WEB-INF/lib folder. > > > > The issue doesn't occur when the server is first started. Only after a period > of 1-2 hours approx. will the next session experience the delay when loading > a jar. This leads me to believe that when the server is started 'knowledge' > of the jars might be cached, however, after a period of inactivity this cache > may expire. The result of this is that the next session after this period > causes the lib folder to be searched again which seems to take the 5-35 > seconds. > > > > My question: Is this theory above correct, and if so, is there a way to > configure Tomcat such that this cache doesn't expire so that it doesn't need > to be re-populated when a new session begins? It depends. Which resources are slow to load? Mark Hi Mark, Doesn't seem to matter which Jar. At first it happened when loading a class from my own custom Jar. I removed this code to
Class loader takes long time after server inactivity - Tomcat Version 9.0.10
Hi Folks, I have a web application that, after a period of inactivity (1-2hours), will stall for 3 – 35 seconds upon the first new session. All subsequent sessions will not experience this delay unless another period of inactivity occurs. I’ve found that the delay occurs when loading jars that live in the WEB-INF/lib folder. The issue doesn’t occur when the server is first started. Only after a period of 1-2 hours approx. will the next session experience the delay when loading a jar. This leads me to believe that when the server is started ‘knowledge’ of the jars might be cached, however, after a period of inactivity this cache may expire. The result of this is that the next session after this period causes the lib folder to be searched again which seems to take the 5-35 seconds. My question: Is this theory above correct, and if so, is there a way to configure Tomcat such that this cache doesn’t expire so that it doesn’t need to be re-populated when a new session begins? Tomcat Version 9.0.10 Any help appreciated, Thanks, Niall
Re: Class loader takes long time after server inactivity - Tomcat Version 9.0.10
Hey Niall, On Tue, 14 Jan 2020 at 13:42, Niall Fitzpatrick wrote: > Hi Folks, > > > > I have a web application that, after a period of inactivity (1-2hours), > will stall for 3 – 35 seconds upon the first new session. All subsequent > sessions will not experience this delay unless another period of inactivity > occurs. > > > > I’ve found that the delay occurs when loading jars that live in the > WEB-INF/lib folder. > > > > The issue doesn’t occur when the server is first started. Only after a > period of 1-2 hours approx. will the next session experience the delay when > loading a jar. This leads me to believe that when the server is started > ‘knowledge’ of the jars might be cached, however, after a period of > inactivity this cache may expire. The result of this is that the next > session after this period causes the lib folder to be searched again which > seems to take the 5-35 seconds. > > > > My question: Is this theory above correct, and if so, is there a way to > configure Tomcat such that this cache doesn’t expire so that it doesn’t > need to be re-populated when a new session begins? > > > > Tomcat Version 9.0.10 > > > > Any help appreciated, > > Thanks, > > Niall > Perhaps others can also correct me if i have missed - Tomcat does have a resource caching and memory settings aka Resources: https://tomcat.apache.org/tomcat-9.0-doc/config/resources.html That said, there is very small possibility (also partly coincidence) that this is impacted by garbage collection cycles. We have got a production scenario where garbage collection interfered with busy hour processing due to having lower memory. But again, that's my own issue. I think the link above may help you configure some things? Regards,
Re: Class loader takes long time after server inactivity - Tomcat Version 9.0.10
On 14/01/2020 14:40, Niall Fitzpatrick wrote: > > > > >> Hi Folks, >> >> >> >> I have a web application that, after a period of inactivity (1-2hours), will >> stall for 3 – 35 seconds upon the first new session. All subsequent sessions >> will not experience this delay unless another period of inactivity occurs. >> >> >> >> I’ve found that the delay occurs when loading jars that live in the >> WEB-INF/lib folder. >> >> >> >> The issue doesn’t occur when the server is first started. Only after a >> period of 1-2 hours approx. will the next session experience the delay when >> loading a jar. This leads me to believe that when the server is started >> ‘knowledge’ of the jars might be cached, however, after a period of >> inactivity this cache may expire. The result of this is that the next >> session after this period causes the lib folder to be searched again which >> seems to take the 5-35 seconds. >> >> >> >> My question: Is this theory above correct, and if so, is there a way to >> configure Tomcat such that this cache doesn’t expire so that it doesn’t need >> to be re-populated when a new session begins? > > It depends. Which resources are slow to load? > > Mark > > Hi Mark, Thanks for the additional info. That helps a lot. Apologies for the following information dump but there are lots of factors to make you aware of. > Doesn't seem to matter which Jar. At first it happened when loading a class > from my own custom Jar. That should be a one-time cost. Once loaded, classes remain in memory until the web application is stopped. Finding the right class can be expensive. The more JARs the application has, the more expensive the process will be. Tomcat does cache the list of entries in a JAR for a short period of time to speed up this search. If the application has been completely idle for a while then you will see a pause on the first attempt to load a class while those caches are refreshed. Tomcat can't hold the cache for too long as it uses memory and locks files. One possible solution is to use a ServletContextListener and load the classes you know will be / are likely to be required when the application starts. > I removed this code to determine if it was the cause. However, then I found > the same problem further downstream. When calling > DocumentBuilderFactory.newInstance() which lives in xml-apis-1.3.04 The delay > of 5-35 seconds occurred here instead. Ah. That will probably be the biggest culprit. DocumentBuilderFactory uses an SPI mechanism to find the correct instance to create. That SPI mechanism involves searching every JAR for a configuration file. The expectation, although it may not be documented as clearly as it should, is that DocumentBuilderFactory.newInstance() is called only once during the lifetime of a web application and the result cached. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Class loader takes long time after server inactivity - Tomcat Version 9.0.10
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Mark, On 1/14/20 9:58 AM, Mark Thomas wrote: > On 14/01/2020 14:40, Niall Fitzpatrick wrote: >> >> >> >> >>> Hi Folks, >>> >>> >>> >>> I have a web application that, after a period of inactivity >>> (1-2hours), will stall for 3 – 35 seconds upon the first new >>> session. All subsequent sessions will not experience this delay >>> unless another period of inactivity occurs. >>> >>> >>> >>> I’ve found that the delay occurs when loading jars that live in >>> the WEB-INF/lib folder. >>> >>> >>> >>> The issue doesn’t occur when the server is first started. Only >>> after a period of 1-2 hours approx. will the next session >>> experience the delay when loading a jar. This leads me to >>> believe that when the server is started ‘knowledge’ of the jars >>> might be cached, however, after a period of inactivity this >>> cache may expire. The result of this is that the next session >>> after this period causes the lib folder to be searched again >>> which seems to take the 5-35 seconds. >>> >>> >>> >>> My question: Is this theory above correct, and if so, is there >>> a way to configure Tomcat such that this cache doesn’t expire >>> so that it doesn’t need to be re-populated when a new session >>> begins? >> >> It depends. Which resources are slow to load? >> >> Mark >> >> Hi Mark, > > Thanks for the additional info. That helps a lot. > > Apologies for the following information dump but there are lots of > factors to make you aware of. > >> Doesn't seem to matter which Jar. At first it happened when >> loading a class from my own custom Jar. > > That should be a one-time cost. Once loaded, classes remain in > memory until the web application is stopped. > > Finding the right class can be expensive. The more JARs the > application has, the more expensive the process will be. Tomcat > does cache the list of entries in a JAR for a short period of time > to speed up this search. If the application has been completely > idle for a while then you will see a pause on the first attempt to > load a class while those caches are refreshed. > > Tomcat can't hold the cache for too long as it uses memory and > locks files. > > One possible solution is to use a ServletContextListener and load > the classes you know will be / are likely to be required when the > application starts. > >> I removed this code to determine if it was the cause. However, >> then I found the same problem further downstream. When calling >> DocumentBuilderFactory.newInstance() which lives in >> xml-apis-1.3.04 The delay of 5-35 seconds occurred here instead. > > Ah. That will probably be the biggest culprit. > DocumentBuilderFactory uses an SPI mechanism to find the correct > instance to create. That SPI mechanism involves searching every JAR > for a configuration file. The expectation, although it may not be > documented as clearly as it should, is that > DocumentBuilderFactory.newInstance() is called only once during the > lifetime of a web application and the result cached. +1 But 5-30 seconds is a looong time. Niall, any chance you have a particularly slow disk, loaded server, or maybe a network share where these files are located? - -chris -BEGIN PGP SIGNATURE- Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl4d7L0ACgkQHPApP6U8 pFi3RA//UMh2x8qb5l2/dPeeq6flT9SfneHpMkrwARaZv2KJbKT6esn1xQOm6bBh Ta2luoS8rIExBbjGujLDU+Ko8yXP7gxBSj+PR9b2wKHiygWDTk/bHcGNbmDkztFA ZyrtyOyO6kq4zcHykZxPxAk5RUAFe1Zk9ni6wVHxwqhKA1QE4luv1T13Q6ju/PXV 89sjxw4JXEnNdcV9BvbllrJ/uDwh7xXiHCrvHFxnT9HYvKNzlCQmEQOnrkLGbc9c 6rL7vQ6bo0ubIx8tL036ZPFvZkWERnedHnxR3eyO3/8Y0PDta+Br3wm/o0PDVcX6 esrUZv2UxXT/JVMrWBZWf3dO8IKSj8Q9WjU+CsRoCmrHrlRo+cuCrPj32n78M8NM CWMqpt/oG1mbjBtLRPWXlwvk8R9/qtQKe+1fpCef6cmVjsUXpFfxNgUHxnjw28Oq HkGdQY2Npf2+S3nepBEVQv1WAJMnwcws/Ue7U4qyQJJrKAceeqUBUi8YEWoFEOum RGQxShWb3Qql47ozfp5q6bcfyZPRpEEa0rYBXpTkF0mlAgi3msxNw0hn6nCEcbsI t+BH1GTba0vwR2bpy/5xXaBuGeMtX8/8AG6ol/QyfngRN2xo2ir1Skc6xtClqMJA 1/8FUUv5g3eTzrz11jNMTBWzbTuCLqYwgkzymT0gfeechI8K/dM= =wQwj -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Class loader takes long time after server inactivity - Tomcat Version 9.0.10
If it helps for reference Here is our singleton...
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.transform.TransformerFactory;
import javax.xml.xpath.XPathFactory;
import javax.xml.xpath.XPathFactoryConfigurationException;
/**
* This class provides a set of methods to obtain a shared instance of various
pre-configured XML parser factories. This
* is intended to avoid searching the classpath when looking for an
implementation of a factory. Using an enumeration to
* create a shared instance guarantees a thread-safe singleton since the
enumeration itself ensures internally that only
* a single instance is available. Each singleton instance is constructed when
the enumeration is referenced for the
* first time. Configuration of a these factories is not expected to be thread
safe; an application programmer should
* not allow an instance of this factory to have its setter methods accessed
from more than one thread.
*/
public final class XMLFactory {
private enum DocumentBuilderFactorySingleton {
SINGLETON;
public final DocumentBuilderFactory instance;
private DocumentBuilderFactorySingleton() {
instance = DocumentBuilderFactory.newInstance();
}
}
private enum TransformerFactorySingleton {
SINGLETON;
public final TransformerFactory instance;
private TransformerFactorySingleton() {
instance = TransformerFactory.newInstance();
}
}
private enum XPathFactorySingleton {
SINGLETON;
public final XPathFactory instance;
private XPathFactorySingleton() {
instance = XPathFactory.newInstance();
}
}
private XMLFactory() {
return;
}
/**
* This method returns a shared instance of a default document builder
factory. Do not call any setter methods on
* this shared instance. If an alternative configuration is necessary, a
new factory should be created.
* @return shared instance of document builder factory
*/
public static final DocumentBuilderFactory
getDocumentBuilderFactoryInstance() {
return DocumentBuilderFactorySingleton.SINGLETON.instance;
}
/**
* This method returns a new instance of a document builder factory.
* @return new instance of document builder factory
*/
public static final DocumentBuilderFactory
newDocumentBuilderFactoryInstance() {
return
DocumentBuilderFactory.newInstance(getDocumentBuilderFactoryInstance().getClass().getName(),
null);
}
/**
* This method returns a shared instance of a default transformer factory.
Do not call any setter methods on this
* shared instance. If an alternative configuration is necessary, a new
factory should be created.
* @return shared instance of transformer factory
*/
public static final TransformerFactory getTransformerFactoryInstance() {
return TransformerFactorySingleton.SINGLETON.instance;
}
/**
* This method returns a new instance of a transformer factory.
* @return new instance of transformer factory
*/
public static final TransformerFactory newTransformerFactoryInstance() {
return
TransformerFactory.newInstance(getTransformerFactoryInstance().getClass().getName(),
null);
}
/**
* This method returns a shared instance of a default xpath factory. Do not
call any setter methods on this shared
* instance. If an alternative configuration is necessary, a new factory
should be created.
* @return shared instance of xpath factory
*/
public static final XPathFactory getXPathFactoryInstance() {
return XPathFactorySingleton.SINGLETON.instance;
}
/**
* This method returns a new instance of a xpath factory.
* @return new instance of xpath factory
* @throws XPathFactoryConfigurationException
*/
public static final XPathFactory newXPathFactoryInstance() throws
XPathFactoryConfigurationException {
return XPathFactory.newInstance(XPathFactory.DEFAULT_OBJECT_MODEL_URI,
getXPathFactoryInstance().getClass().getName(), null);
}
}
-Original Message-
From: Niall Fitzpatrick
Sent: Tuesday, January 14, 2020 11:48 AM
To: Tomcat Users List
Subject: Re: Class loader takes long time after server inactivity - Tomcat
Version 9.0.10
From: Christopher Schultz
Sent: 14 January 2020 16:30
To: users@tomcat.apache.org
Subject: Re: Class loader takes long time after server inactivity - Tomcat
Version 9.0.10
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
Mark,
On 1/14/20 9:58 AM, Mark Thomas wrote:
> On 14/01/2020 14:40, Niall Fitzpatrick wrote:
>>
>>
>>
>>
>>> Hi Folks,
>>>
>>>
>>>
>>> I have a web application that, after a period of inactivity
>>> (1-2hours), will stall for 3 - 35 seconds upon the first new
>>> session. All subsequent sessions will not experience this delay
>>>
Re: Class loader takes long time after server inactivity - Tomcat Version 9.0.10
From: Christopher Schultz Sent: 14 January 2020 16:30 To: users@tomcat.apache.org Subject: Re: Class loader takes long time after server inactivity - Tomcat Version 9.0.10 -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Mark, On 1/14/20 9:58 AM, Mark Thomas wrote: > On 14/01/2020 14:40, Niall Fitzpatrick wrote: >> >> >> >> >>> Hi Folks, >>> >>> >>> >>> I have a web application that, after a period of inactivity >>> (1-2hours), will stall for 3 – 35 seconds upon the first new >>> session. All subsequent sessions will not experience this delay >>> unless another period of inactivity occurs. >>> >>> >>> >>> I’ve found that the delay occurs when loading jars that live in >>> the WEB-INF/lib folder. >>> >>> >>> >>> The issue doesn’t occur when the server is first started. Only >>> after a period of 1-2 hours approx. will the next session >>> experience the delay when loading a jar. This leads me to >>> believe that when the server is started ‘knowledge’ of the jars >>> might be cached, however, after a period of inactivity this >>> cache may expire. The result of this is that the next session >>> after this period causes the lib folder to be searched again >>> which seems to take the 5-35 seconds. >>> >>> >>> >>> My question: Is this theory above correct, and if so, is there >>> a way to configure Tomcat such that this cache doesn’t expire >>> so that it doesn’t need to be re-populated when a new session >>> begins? >> >> It depends. Which resources are slow to load? >> >> Mark >> >> Hi Mark, > > Thanks for the additional info. That helps a lot. > > Apologies for the following information dump but there are lots of > factors to make you aware of. > >> Doesn't seem to matter which Jar. At first it happened when >> loading a class from my own custom Jar. > > That should be a one-time cost. Once loaded, classes remain in > memory until the web application is stopped. > > Finding the right class can be expensive. The more JARs the > application has, the more expensive the process will be. Tomcat > does cache the list of entries in a JAR for a short period of time > to speed up this search. If the application has been completely > idle for a while then you will see a pause on the first attempt to > load a class while those caches are refreshed. > > Tomcat can't hold the cache for too long as it uses memory and > locks files. > > One possible solution is to use a ServletContextListener and load > the classes you know will be / are likely to be required when the > application starts. > >> I removed this code to determine if it was the cause. However, >> then I found the same problem further downstream. When calling >> DocumentBuilderFactory.newInstance() which lives in >> xml-apis-1.3.04 The delay of 5-35 seconds occurred here instead. > > Ah. That will probably be the biggest culprit. > DocumentBuilderFactory uses an SPI mechanism to find the correct > instance to create. That SPI mechanism involves searching every JAR > for a configuration file. The expectation, although it may not be > documented as clearly as it should, is that > DocumentBuilderFactory.newInstance() is called only once during the > lifetime of a web application and the result cached. +1 But 5-30 seconds is a looong time. Niall, any chance you have a particularly slow disk, loaded server, or maybe a network share where these files are located? - -chris Hi Chris, I don't believe so. A customer reported the issue, and I was able to recreate it on our own test environment. I will say this - the web application has 263 jars in it's lib folder. That's got to be a lot of searching. Thanks for everyone's suggestions. I'm going to try the wrapper class solution where I can cache the result. I'll send an update with what I learned. Niall - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Encoding properties
On 14/01/2020 19:50, Luis Arriaga wrote:
> Hi Apache team,
Please don't cross-post. This discussion can continue on the dev list
since you posted there first and it is related to changing the source code.
> We are updating our encoding implementation on Tomcat and the ServiceNow
> platform from ISO-8859-1 to UTF-8 and ran into some concerns. Our Tomcat
> setup includes 8.5.47 on a CentOS VM, where Tomcat serves as our Java Servlet
> Web Server and we terminate SSL on our load balancer.
>
> There is a URIEncoding property that defaults to UTF-8 if it is not
> specified. Is there any reason there is no BodyEncoding property or is there
> a workaround you guys
Some people read "guys" as being gender specific and, as such, it is not
appropriate to use that term when referring to the community as a whole.
Mark
> are aware of that does not require the source code to be modified? Looking
> through the Tomcat source code the default body encoding seems to be
> ISO-8859-1, looking at the Parameters, ByteChunk, and Constants classes there
> are two variables DEFAULT_BODY_CHARSET and DEFAULT_CHARSET that determine the
> body charset/encoding.
>
> We have forked the Tomcat source code and applied the changes below which
> fixed the issue. Are you guys aware of this? Seems strange to have a
> URIEncoding property but not a BodyEncoding property unless I am missing
> something. Maybe this is an enhancement request we can submit unless there is
> a valid reason to not have such property.
>
> diff --git a/java/org/apache/coyote/Constants.java
> b/java/org/apache/coyote/Constants.java
> index 9de194d55..0883904f4 100644
> --- a/java/org/apache/coyote/Constants.java
> +++ b/java/org/apache/coyote/Constants.java
> @@ -33,7 +33,7 @@ public final class Constants {
> public static final String DEFAULT_CHARACTER_ENCODING="ISO-8859-1";
> public static final Charset DEFAULT_URI_CHARSET =
> StandardCharsets.ISO_8859_1;
> -public static final Charset DEFAULT_BODY_CHARSET =
> StandardCharsets.ISO_8859_1;
> +public static final Charset DEFAULT_BODY_CHARSET =
> StandardCharsets.UTF_8;
> public static final int MAX_NOTES = 32;
> diff --git a/java/org/apache/tomcat/util/buf/ByteChunk.java
> b/java/org/apache/tomcat/util/buf/ByteChunk.java
> index 555c0f6b8..ed9f6e5ea 100644
> --- a/java/org/apache/tomcat/util/buf/ByteChunk.java
> +++ b/java/org/apache/tomcat/util/buf/ByteChunk.java
> @@ -123,7 +123,7 @@ public final class ByteChunk extends AbstractChunk {
> * standards seem to converge, but the servlet API requires 8859_1, and
> this
> * object is used mostly for servlets.
> */
> -public static final Charset DEFAULT_CHARSET =
> StandardCharsets.ISO_8859_1;
> +public static final Charset DEFAULT_CHARSET = StandardCharsets.UTF_8;
> private transient Charset charset;
> diff --git a/java/org/apache/tomcat/util/http/Parameters.java
> b/java/org/apache/tomcat/util/http/Parameters.java
> index 4d7d6cc1e..f59f75514 100644
> --- a/java/org/apache/tomcat/util/http/Parameters.java
> +++ b/java/org/apache/tomcat/util/http/Parameters.java
> @@ -266,7 +266,7 @@ public final class Parameters {
> */
> @Deprecated
> public static final String DEFAULT_ENCODING = "ISO-8859-1";
> -private static final Charset DEFAULT_BODY_CHARSET =
> StandardCharsets.ISO_8859_1;
> +private static final Charset DEFAULT_BODY_CHARSET =
> StandardCharsets.UTF_8;
> private static final Charset DEFAULT_URI_CHARSET =
> StandardCharsets.UTF_8;
>
>
>
> _
> Luis Arriaga
> Software Engineer
> M: +17605192599
> servicenow.com
> LinkedIn |
> Twitter |
> YouTube |
> Facebook
>
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Encoding properties
Hi Apache team,
We are updating our encoding implementation on Tomcat and the ServiceNow
platform from ISO-8859-1 to UTF-8 and ran into some concerns. Our Tomcat setup
includes 8.5.47 on a CentOS VM, where Tomcat serves as our Java Servlet Web
Server and we terminate SSL on our load balancer.
There is a URIEncoding property that defaults to UTF-8 if it is not specified.
Is there any reason there is no BodyEncoding property or is there a workaround
you guys are aware of that does not require the source code to be modified?
Looking through the Tomcat source code the default body encoding seems to be
ISO-8859-1, looking at the Parameters, ByteChunk, and Constants classes there
are two variables DEFAULT_BODY_CHARSET and DEFAULT_CHARSET that determine the
body charset/encoding.
We have forked the Tomcat source code and applied the changes below which fixed
the issue. Are you guys aware of this? Seems strange to have a URIEncoding
property but not a BodyEncoding property unless I am missing something. Maybe
this is an enhancement request we can submit unless there is a valid reason to
not have such property.
diff --git a/java/org/apache/coyote/Constants.java
b/java/org/apache/coyote/Constants.java
index 9de194d55..0883904f4 100644
--- a/java/org/apache/coyote/Constants.java
+++ b/java/org/apache/coyote/Constants.java
@@ -33,7 +33,7 @@ public final class Constants {
public static final String DEFAULT_CHARACTER_ENCODING="ISO-8859-1";
public static final Charset DEFAULT_URI_CHARSET =
StandardCharsets.ISO_8859_1;
-public static final Charset DEFAULT_BODY_CHARSET =
StandardCharsets.ISO_8859_1;
+public static final Charset DEFAULT_BODY_CHARSET = StandardCharsets.UTF_8;
public static final int MAX_NOTES = 32;
diff --git a/java/org/apache/tomcat/util/buf/ByteChunk.java
b/java/org/apache/tomcat/util/buf/ByteChunk.java
index 555c0f6b8..ed9f6e5ea 100644
--- a/java/org/apache/tomcat/util/buf/ByteChunk.java
+++ b/java/org/apache/tomcat/util/buf/ByteChunk.java
@@ -123,7 +123,7 @@ public final class ByteChunk extends AbstractChunk {
* standards seem to converge, but the servlet API requires 8859_1, and
this
* object is used mostly for servlets.
*/
-public static final Charset DEFAULT_CHARSET = StandardCharsets.ISO_8859_1;
+public static final Charset DEFAULT_CHARSET = StandardCharsets.UTF_8;
private transient Charset charset;
diff --git a/java/org/apache/tomcat/util/http/Parameters.java
b/java/org/apache/tomcat/util/http/Parameters.java
index 4d7d6cc1e..f59f75514 100644
--- a/java/org/apache/tomcat/util/http/Parameters.java
+++ b/java/org/apache/tomcat/util/http/Parameters.java
@@ -266,7 +266,7 @@ public final class Parameters {
*/
@Deprecated
public static final String DEFAULT_ENCODING = "ISO-8859-1";
-private static final Charset DEFAULT_BODY_CHARSET =
StandardCharsets.ISO_8859_1;
+private static final Charset DEFAULT_BODY_CHARSET = StandardCharsets.UTF_8;
private static final Charset DEFAULT_URI_CHARSET = StandardCharsets.UTF_8;
_
Luis Arriaga
Software Engineer
M: +17605192599
servicenow.com
LinkedIn |
Twitter |
YouTube |
Facebook
