Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool
Hi folks, I recently enabled a QueryTimeoutInterceptor with queryTimeout of 60 seconds in a JDBC Pool data source (7.0.37). When the app was shut down, Tomcat said: The web application [/...] appears to have started a thread named [OracleTimeoutPollingThread] but has failed to stop it... We are using Oracle 11.2g with 11.2.0.3 JDBC drivers. I have figured out that this thread is spawned by the driver itself. According to this Stackoverflow answer [1] this is a long-living thread, same says the JDBC FAQ [2] of Oracle. The thread seems to work like Pool's PoolCleaner thread. A few month a ago I reported the same issue with the PoolCleaner thread and Filip fixed the class loader orders. Can this be a false-positive by the memory leak detector since this thread lives only once in the entire VM? Thanks, Michael [1] http://stackoverflow.com/a/3387312/696632 [2] http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-faq-090281.html#02_14 - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool
On 07/05/2013 09:16, Michael-O wrote: Hi folks, I recently enabled a QueryTimeoutInterceptor with queryTimeout of 60 seconds in a JDBC Pool data source (7.0.37). When the app was shut down, Tomcat said: The web application [/...] appears to have started a thread named [OracleTimeoutPollingThread] but has failed to stop it... We are using Oracle 11.2g with 11.2.0.3 JDBC drivers. I have figured out that this thread is spawned by the driver itself. According to this Stackoverflow answer [1] this is a long-living thread, same says the JDBC FAQ [2] of Oracle. The thread seems to work like Pool's PoolCleaner thread. A few month a ago I reported the same issue with the PoolCleaner thread and Filip fixed the class loader orders. Can this be a false-positive by the memory leak detector since this thread lives only once in the entire VM? No. It is a memory leak and either or bug in your application or in the JDBC driver. Where is the Oracle JDBC driver? CATALINA_[BASE|HOME]/lib or WEB-INF/lib Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Aw: Re: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool
Von: Mark Thomas ma...@apache.org On 07/05/2013 09:16, Michael-O wrote: Hi folks, I recently enabled a QueryTimeoutInterceptor with queryTimeout of 60 seconds in a JDBC Pool data source (7.0.37). When the app was shut down, Tomcat said: The web application [/...] appears to have started a thread named [OracleTimeoutPollingThread] but has failed to stop it... We are using Oracle 11.2g with 11.2.0.3 JDBC drivers. I have figured out that this thread is spawned by the driver itself. According to this Stackoverflow answer [1] this is a long-living thread, same says the JDBC FAQ [2] of Oracle. The thread seems to work like Pool's PoolCleaner thread. A few month a ago I reported the same issue with the PoolCleaner thread and Filip fixed the class loader orders. Can this be a false-positive by the memory leak detector since this thread lives only once in the entire VM? No. It is a memory leak and either or bug in your application or in the JDBC driver. Where is the Oracle JDBC driver? CATALINA_[BASE|HOME]/lib or WEB-INF/lib The driver is in the $CATALINA_HOME/lib only where $CATALINA_BASE != $CATALINA_HOME. This was done for a single webapp for testing purposes. Does this make a difference? How do you know that this is not a false-positive? If you really know for sure, I can open a service request with Oracle Support. Michael - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Aw: Re: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool
On 07/05/2013 10:25, Michael-O wrote: Von: Mark Thomas ma...@apache.org On 07/05/2013 09:16, Michael-O wrote: Hi folks, I recently enabled a QueryTimeoutInterceptor with queryTimeout of 60 seconds in a JDBC Pool data source (7.0.37). When the app was shut down, Tomcat said: The web application [/...] appears to have started a thread named [OracleTimeoutPollingThread] but has failed to stop it... We are using Oracle 11.2g with 11.2.0.3 JDBC drivers. I have figured out that this thread is spawned by the driver itself. According to this Stackoverflow answer [1] this is a long-living thread, same says the JDBC FAQ [2] of Oracle. The thread seems to work like Pool's PoolCleaner thread. A few month a ago I reported the same issue with the PoolCleaner thread and Filip fixed the class loader orders. Can this be a false-positive by the memory leak detector since this thread lives only once in the entire VM? No. It is a memory leak and either or bug in your application or in the JDBC driver. Where is the Oracle JDBC driver? CATALINA_[BASE|HOME]/lib or WEB-INF/lib The driver is in the $CATALINA_HOME/lib only where $CATALINA_BASE != $CATALINA_HOME. This was done for a single webapp for testing purposes. Does this make a difference? The important thing is that it isn't in WEB-INF/lib. How do you know that this is not a false-positive? Experience, a lot of research into memory leaks and I wrote Tomcat's memory leak detection code. If you really know for sure, I can open a service request with Oracle Support. Good luck with that. The problem is that when the driver creates the thread it does so when the current class loader is the web application class loader. That means that the Thread will be created with a context class loader set to the web application class loader. That will trigger a memory leak when the web application is stopped because a reference is retained to the web application's class loader. What the driver should do is, after it creates the thread, set the thread's context class loader to the class loader that loaded the driver. What you are seeing is a variation of the leak described on page 15 of [1]. Mark [1] http://people.apache.org/~markt/presentations/2010-11-04-Memory-Leaks-60mins.pdf - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Attacks in Apache servers
Hi All, Thanks for your views. It seems to be like very difficult to come for a conclusion :-) Regards Eashwar On Thu, May 2, 2013 at 11:08 PM, David N. Smith david.sm...@cornell.eduwrote: Didn't you know that 'rm' was vulnerable on Linux?!?! An attacker with escalated privileges can -- through clever use of this misunderstood command with code so complicated, that this enormous vulnerability went unnoticed for decades -- wreak havoc on any Linux system connected to the iterwebs. The only plausible mitigation of this egregious vulnerability is to uninstall the 'rm' package or switch to a more secure OS. I think the vulnerability is limited to versions that support the options -r and -f. ;-) -- David - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Re: Re: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool
Von: Mark Thomas ma...@apache.org On 07/05/2013 10:25, Michael-O wrote: Von: Mark Thomas ma...@apache.org On 07/05/2013 09:16, Michael-O wrote: Hi folks, I recently enabled a QueryTimeoutInterceptor with queryTimeout of 60 seconds in a JDBC Pool data source (7.0.37). When the app was shut down, Tomcat said: The web application [/...] appears to have started a thread named [OracleTimeoutPollingThread] but has failed to stop it... We are using Oracle 11.2g with 11.2.0.3 JDBC drivers. I have figured out that this thread is spawned by the driver itself. According to this Stackoverflow answer [1] this is a long-living thread, same says the JDBC FAQ [2] of Oracle. The thread seems to work like Pool's PoolCleaner thread. A few month a ago I reported the same issue with the PoolCleaner thread and Filip fixed the class loader orders. Can this be a false-positive by the memory leak detector since this thread lives only once in the entire VM? No. It is a memory leak and either or bug in your application or in the JDBC driver. Where is the Oracle JDBC driver? CATALINA_[BASE|HOME]/lib or WEB-INF/lib The driver is in the $CATALINA_HOME/lib only where $CATALINA_BASE != $CATALINA_HOME. This was done for a single webapp for testing purposes. Does this make a difference? The important thing is that it isn't in WEB-INF/lib. How do you know that this is not a false-positive? Experience, a lot of research into memory leaks and I wrote Tomcat's memory leak detection code. If you really know for sure, I can open a service request with Oracle Support. Good luck with that. The problem is that when the driver creates the thread it does so when the current class loader is the web application class loader. That means that the Thread will be created with a context class loader set to the web application class loader. That will trigger a memory leak when the web application is stopped because a reference is retained to the web application's class loader. What the driver should do is, after it creates the thread, set the thread's context class loader to the class loader that loaded the driver. What you are seeing is a variation of the leak described on page 15 of [1]. After reading the slides and your explanation this makes sense. It's the same issue as Pool cleaner thread should be created using the classloader that loaded the pool, not the context loader (fhanik) fixed in 7.0.27. I will file SR and let you know. Michael - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Aw: Re: Re: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool
Von: Mark Thomas ma...@apache.org On 07/05/2013 10:25, Michael-O wrote: Von: Mark Thomas ma...@apache.org On 07/05/2013 09:16, Michael-O wrote: Hi folks, I recently enabled a QueryTimeoutInterceptor with queryTimeout of 60 seconds in a JDBC Pool data source (7.0.37). When the app was shut down, Tomcat said: The web application [/...] appears to have started a thread named [OracleTimeoutPollingThread] but has failed to stop it... We are using Oracle 11.2g with 11.2.0.3 JDBC drivers. I have figured out that this thread is spawned by the driver itself. According to this Stackoverflow answer [1] this is a long-living thread, same says the JDBC FAQ [2] of Oracle. The thread seems to work like Pool's PoolCleaner thread. A few month a ago I reported the same issue with the PoolCleaner thread and Filip fixed the class loader orders. Can this be a false-positive by the memory leak detector since this thread lives only once in the entire VM? No. It is a memory leak and either or bug in your application or in the JDBC driver. Where is the Oracle JDBC driver? CATALINA_[BASE|HOME]/lib or WEB-INF/lib The driver is in the $CATALINA_HOME/lib only where $CATALINA_BASE != $CATALINA_HOME. This was done for a single webapp for testing purposes. Does this make a difference? The important thing is that it isn't in WEB-INF/lib. How do you know that this is not a false-positive? Experience, a lot of research into memory leaks and I wrote Tomcat's memory leak detection code. If you really know for sure, I can open a service request with Oracle Support. Good luck with that. SR created. I will keep you updated. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Certified JDKs for Tomcat 7.0.35
Hello What are the certified JDKs that Tomcat 7.0.35 can run on in a production environment? Hendré - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Certified JDKs for Tomcat 7.0.35
Hendré Louw wrote: Hello What are the certified JDKs that Tomcat 7.0.35 can run on in a production environment? I believe that should be mentioned on the which version ? page at http://tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Certified JDKs for Tomcat 7.0.35
On 07/05/2013 12:47, André Warnier wrote: Hendré Louw wrote: Hello What are the certified JDKs that Tomcat 7.0.35 can run on in a production environment? I believe that should be mentioned on the which version ? page at http://tomcat.apache.org +1. Those are the minimum versions. Given the almost EOL state of Java 1.6 you should be running Tomcat (any version) on the latest available Java 7 release for your platform. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Certified JDKs for Tomcat 7.0.35
André Thank you for the link. The page states Minimum Java Version. Can I therefor run Tomcat 7.0.35 on Java 1.7.0_21? Is the exact Java version the Tomcat development team used for a specific Tomcat release not more correct for production environments? Hendré On 7 May 2013 13:47, André Warnier a...@ice-sa.com wrote: Hendré Louw wrote: Hello What are the certified JDKs that Tomcat 7.0.35 can run on in a production environment? I believe that should be mentioned on the which version ? page at http://tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org -- Hendré - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Certified JDKs for Tomcat 7.0.35
Hendré Louw wrote: André Thank you for the link. The page states Minimum Java Version. Can I therefor run Tomcat 7.0.35 on Java 1.7.0_21? Is the exact Java version the Tomcat development team used for a specific Tomcat release not more correct for production environments? It should not be, but I'll let someone more qualified than I, give a more precise answer to that. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Certified JDKs for Tomcat 7.0.35
On 07/05/2013 13:43, André Warnier wrote: Hendré Louw wrote: André Thank you for the link. The page states Minimum Java Version. Can I therefor run Tomcat 7.0.35 on Java 1.7.0_21? Is the exact Java version the Tomcat development team used for a specific Tomcat release not more correct for production environments? It should not be, +1. Any Java 6 or Java 7 release will work. but I'll let someone more qualified than I, give a more precise answer to that. The best choice is the latest Java 7 release for your platform. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Why is context.xml no longer copied to Catalina/localhost/myapp.xml?
On Mon, May 06, 2013 at 04:35:19PM -0400, Jesse Barnum wrote: [snip] I am sure that this would be out of scope, but if we pictured an ideal scenario, it seems like there would be one configuration file that is tightly managed by the developer, which is replaced when the app is redeployed, and a different configuration file that is intended for end user customization, which is stored separately. Well, the developer can simply pack into the app. whatever internal configuration is needed, since he has ready access to the interior of the app and can deposit on the classpath *.properties, *.xml, or anything else he wants. He can have no certain knowledge of the app's runtime environment and should not assume, only specify requirements, and provide sensible defaults when there are some. The deployer, OTOH, has ready access to the app's environment, including its Context, but should not be assumed to have such access to the interior of the app. So this division of labor depends on the developer's discipline in distinguishing internal vs. external configuration and coding the app. to look in the proper place for each. I don't see a good way for the container to make up for incorrect design in this area. -- Mark H. Wood, Lead System Programmer mw...@iupui.edu Machines should not be friendly. Machines should be obedient. signature.asc Description: Digital signature
Re: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
Michael,
On 5/7/13 6:43 AM, Michael-O wrote:
Von: Mark Thomas ma...@apache.org On 07/05/2013 10:25,
Michael-O wrote:
Von: Mark Thomas ma...@apache.org On 07/05/2013 09:16,
Michael-O wrote:
Hi folks,
I recently enabled a QueryTimeoutInterceptor with
queryTimeout of 60 seconds in a JDBC Pool data source
(7.0.37). When the app was shut down, Tomcat said: The web
application [/...] appears to have started a thread named
[OracleTimeoutPollingThread] but has failed to stop it...
We are using Oracle 11.2g with 11.2.0.3 JDBC drivers. I
have figured out that this thread is spawned by the driver
itself. According to this Stackoverflow answer [1] this is
a long-living thread, same says the JDBC FAQ [2] of
Oracle.
The thread seems to work like Pool's PoolCleaner thread. A
few month a ago I reported the same issue with the
PoolCleaner thread and Filip fixed the class loader
orders.
Can this be a false-positive by the memory leak detector
since this thread lives only once in the entire VM?
No. It is a memory leak and either or bug in your application
or in the JDBC driver.
Where is the Oracle JDBC driver? CATALINA_[BASE|HOME]/lib or
WEB-INF/lib
The driver is in the $CATALINA_HOME/lib only where
$CATALINA_BASE != $CATALINA_HOME. This was done for a single
webapp for testing purposes.
Does this make a difference?
The important thing is that it isn't in WEB-INF/lib.
How do you know that this is not a false-positive?
Experience, a lot of research into memory leaks and I wrote
Tomcat's memory leak detection code.
If you really know for sure, I can open a service request with
Oracle Support.
Good luck with that.
The problem is that when the driver creates the thread it does so
when the current class loader is the web application class
loader. That means that the Thread will be created with a context
class loader set to the web application class loader. That will
trigger a memory leak when the web application is stopped because
a reference is retained to the web application's class loader.
What the driver should do is, after it creates the thread, set
the thread's context class loader to the class loader that loaded
the driver.
What you are seeing is a variation of the leak described on page
15 of [1].
After reading the slides and your explanation this makes sense.
It's the same issue as Pool cleaner thread should be created using
the classloader that loaded the pool, not the context loader
(fhanik) fixed in 7.0.27.
I will file SR and let you know.
Note that you might be able to write your own code to mitigate this
problem, depending on exactly when that thread is created. If the
timeout thread isn't created until you actually try to issue a query
with a timeout, try something like this in a ServletContextListener's
contextInitialized method:
// NOTE: No resource management is being done in this example
// Get the current ClassLoader -- should be WebappClassLoader
ClassLoader cl = Thread.currentThread().getContextClassLoader();
// WebappClassLoader.getParent should be common loader
Thread.currentThread().setContextClassLoader(cl.getParent());
try
{
Connection conn = ...; // However you get a connection
Statement s = conn.createStatement();
s.setQueryTimeout(5000); // Doesn't really matter what TO is
s.execute(SELECT 1 FROM dual, );
}
finally
{
// Pop back to the original ClassLoader
Thread.currentThread().setContextClassLoader(cl);
}
This is the exact technique that Tomcat's
JreMemoryLeakPreventionListener uses to prevent ClassLoader leaks. A
couple of notes:
1. This won't work under a SecurityManager. If you need to operate
under a SecurityManager, have a look at the
JreMemoryLeakPreventionListener code and adapt it to the above.
2. If the Oracle driver launches the thread when the DataSource is
created, it might happen too early for a ServletContextListener to
intervene. In that case, simply modify the
JreMEmoryLeakPreventionListener code directly. Patches are always welcome.
3. If Oracle fixes this bug, Tomcat should not prevent against it. If
Oracle refuses to acknowledge/fix/etc. this bug, then it may make
sense to include such code in JreMemoryLeakPreventionListener, but it
should probably be done in such a way that a) it's not enabled by
default and b) the query used for triggering the Thread to be created
is user-selectable with maybe a reasonable default (like SELECT 1
FROM dual, as that tends to be valid in most RDBMSs).
- -chris
-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJRiRulAAoJEBzwKT+lPKRYQCUP/jOhPNIje1C114EOvDkGVb7T
jyx+LiULlos6ZE3eMlu889tC7rjRZVRp39TEs8qMW42JT6etLAeY2bwWtqenbM7m
Re: Certified JDKs for Tomcat 7.0.35
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hendré, On 5/7/13 7:41 AM, Hendré Louw wrote: What are the certified JDKs that Tomcat 7.0.35 can run on in a production environment? I run Tomcat 7.0.3x+ on Oracle Java 1.6.0_26-b03. I haven't quite moved-up to Java 1.7 anywhere, yet, but Mark is entirely right: 1.6 is about EOL so it's time. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJRiRxSAAoJEBzwKT+lPKRYpwIQALbdNBt0hWrwbI9pTa8lPx4c Z6CjH90m4D5v7dLer1vDa5AMMgs2vuEbCJTO7CTZk+EZSwDp12Ninm7ItWo80Uci VQOOJkwuYkoe1Z90MrvlU1YijJTVjNDF3H1EZNd5NusdIvEvHdK1kR3pfBEBETkk /2FOwmbDGcjDt9EdxEMiA/qs/yAcnzsful/FWjvtbIh9XITF+uNBsSO9pTMYRP0x vVzOVaP9GyGTOozN507upnO8ExeDqKS46kDTkEtzh7tpT2KECul1xr6soFggacKa wO4kinfQQADaQ0FOO/HR86FvaM1CBmPGzf8r4BzuMssplm/82nzRERzytJy5YM1z EA43K6pQ+mpkPw8LZehtRbilIu1R7wqXpwqKWlVmWMKvTon6dqnQqmsAPq6V9ipt dJUD1Ix2gF/BpsAPv6ScLGzcM0p6ib69FpSGsd0ZYcuxOfuXzSM1k77tFHMlcY7u a4u/6hQXLUZORtsZaKCIb2aoVAORju42fO0SKByMvTMyhzCLkGsN4FJgI1daGvPJ u1bpvA2VHCpDVkx09aKPzLriz5cC24jzyEucbBjLRaM4yv1ffZsvaf5wEGYoU2mx d9W6Os/+kQl+Tv3qcnFUFZy7ybP6jvmGlHCsl9WvsNVT1AauUP+QLXQmX8TYhMgR besxlmvuZTEsdPvzMXiR =x4NF -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
server jvm
Installing Tomcat 7.0.39 using the service.bat will pick up the server jvm.dll located in the jdk directory, but now that there is a separate download for a server jre at 7u21, should we be using that instead? C:\Program Files\Java\jdk1.7.0_21\jre\bin\server\jvm.dll Or use the new Server JRE at 7u21 that is now a separate download. Leo
RE: server jvm
From: Leo Donahue - RDSA IT [mailto:leodona...@mail.maricopa.gov] Subject: server jvm now that there is a separate download for a server jre at 7u21, should we be using that instead? It looks like the Server JRE is somewhere in between a JDK and traditional JRE. It is 64-bit only, but includes some of the tools normally present only in the JDK download. As far as the JVM itself, the 64-bit server versions are identical in all the downloads. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Why is context.xml no longer copied to Catalina/localhost/myapp.xml?
-Original Message- From: Mark Thomas [mailto:ma...@apache.org] Sent: Monday, May 06, 2013 4:01 PM To: Tomcat Users List Subject: Re: Why is context.xml no longer copied to Catalina/localhost/myapp.xml? On 06/05/2013 21:35, Jesse Barnum wrote: On May 6, 2013, at 1:55 PM, Mark Thomas ma...@apache.org wrote: Right now, probably not. There are a couple of issues in this area (the thread I referenced, unpacking WARs outside the appBase into the appBase, lack of clarity on exactly what the expected behaviour in any given scenario) that I am actively working on. My rough plan is: - document what I think should happen (as simply as possible - this is proving to be the hard part as there are so many variables) - present this to the community for discussion / feedback - implement it for 8.0.x - probably back-port it to 7.0.x For you particular scenario I am considering allowing per Context override of copyXML that would enable your app to work as you want in a default Tomcat 7 install. Every new option, however adds to the overall complexity so I am still working through this. Watch this space. Mark Make sense - thanks Mark. I am sure that this would be out of scope, but if we pictured an ideal scenario, it seems like there would be one configuration file that is tightly managed by the developer, which is replaced when the app is redeployed, and a different configuration file that is intended for end user customization, which is stored separately. The way to do that is to keep copyXML=false, parameterise [1] the META- INF/context.xml and then specify the necessary parameter values in catalina.properties. That way the developer is free to manage META-INF/context.xml but any updates won't change the parameterised values. If the app doesn't fail to start if the parameters are set, it should be easy to add a ServletContextListener to make sure that it does. Mark [1] http://tomcat.apache.org/tomcat-7.0-doc/config/index.html (2nd para) Which won't necessarily work for those of us who deploy into multi-tenant (multi-host) environments. For example, each context.xml might have a DB resource that has to be customized differently for each tenant/host. We certainly don't want to have to modify each context.xml with every new version that is produced, and they certainly could not use the same parameter for substitution. In that case, it's a one and done scenario, unless there are mods to the context.xml that are required by the new app release. Just one of those may varying scenarios you were talking about. Jeff - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Why is context.xml no longer copied to Catalina/localhost/myapp.xml?
-Original Message- From: Mark H. Wood [mailto:mw...@iupui.edu] Sent: Tuesday, May 07, 2013 8:41 AM To: users@tomcat.apache.org Subject: Re: Why is context.xml no longer copied to Catalina/localhost/myapp.xml? On Mon, May 06, 2013 at 04:35:19PM -0400, Jesse Barnum wrote: [snip] I am sure that this would be out of scope, but if we pictured an ideal scenario, it seems like there would be one configuration file that is tightly managed by the developer, which is replaced when the app is redeployed, and a different configuration file that is intended for end user customization, which is stored separately. Well, the developer can simply pack into the app. whatever internal configuration is needed, since he has ready access to the interior of the app and can deposit on the classpath *.properties, *.xml, or anything else he wants. He can have no certain knowledge of the app's runtime environment and should not assume, only specify requirements, and provide sensible defaults when there are some. The deployer, OTOH, has ready access to the app's environment, including its Context, but should not be assumed to have such access to the interior of the app. So this division of labor depends on the developer's discipline in distinguishing internal vs. external configuration and coding the app. to look in the proper place for each. I don't see a good way for the container to make up for incorrect design in this area. That's perpetual dilemma for those of us who develop our apps for commercial distribution. We usually don't have access to a lot of needed information about the deployment environment, and really don't want to know that much. We don't normally have things like machine names, database names, schema names and passwords, etc., and our customers are usually reluctant to provide that information to us, at least prior to on-site install (if there is one). So that sort of stuff has to be in a location that it is relatively easy to point the SysAdmin towards and tell him what should be modified (or done via an install script, where possible). And in any case, I don't want to have to modify several hundred context.xml or properties files, etc. and then generate individual war files for each customer. I already have my hands full just creating the half-dozen variations of our product each release. Jeff - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Why is context.xml no longer copied to Catalina/localhost/myapp.xml?
On May 7, 2013, at 9:40 AM, Mark H. Wood mw...@iupui.edu wrote: Well, the developer can simply pack into the app. whatever internal configuration is needed, since he has ready access to the interior of the app and can deposit on the classpath *.properties, *.xml, or anything else he wants. He can have no certain knowledge of the app's runtime environment and should not assume, only specify requirements, and provide sensible defaults when there are some. The deployer, OTOH, has ready access to the app's environment, including its Context, but should not be assumed to have such access to the interior of the app. So this division of labor depends on the developer's discipline in distinguishing internal vs. external configuration and coding the app. to look in the proper place for each. I don't see a good way for the container to make up for incorrect design in this area. -- Mark H. Wood, Lead System Programmer mw...@iupui.edu Machines should not be friendly. Machines should be obedient. Mark, can you give me an example of a use case where it is useful for the deployer to modify parameter values in the META-INF/context.xml file? Assume that at some point, a new version of the application will be deployed, and also assume that the deployer does not wish to re-apply the same customizations with each release. Without getting into the pros and cons of your first paragraph (which places all responsibility for managing app preferences on the developer), would you agree that the current approach (leaving the context.xml file in the web app) is not fulfilling one of its intended purposes, which is allowing the deployer to customize the application behavior? --Jesse Barnum, President, 360Works http://www.360works.com
Re: [OT] Certified JDKs for Tomcat 7.0.35
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Mark, On 5/7/13 8:48 AM, Mark Thomas wrote: On 07/05/2013 13:43, André Warnier wrote: Hendré Louw wrote: André Thank you for the link. The page states Minimum Java Version. Can I therefor run Tomcat 7.0.35 on Java 1.7.0_21? Is the exact Java version the Tomcat development team used for a specific Tomcat release not more correct for production environments? It should not be, +1. Any Java 6 or Java 7 release will work. but I'll let someone more qualified than I, give a more precise answer to that. The best choice is the latest Java 7 release for your platform. I'm interested in the community's experiences with OpenJDK 7. We use Debian Linux and Oracle's JVM is not available at the 1.7-level through their package-management system: only OpenJDK is available. Can anyone comment on the stability and suitability of OpenJDK 1.7 in production systems? The current patch level available via Debian's apt is 7u3. Thanks, - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJRiUmYAAoJEBzwKT+lPKRYLfAP/j3MwIghviEri5yA8SJZca+g 7gLxN6MsswfpZF9hpKuyfNeqHnCNz/8o9gbyqFGZtEiDQRfwUFVLOvGTTsTlQphx l4NUmIeIeWg5inctIuyNLRAf+7V6pdAt+eqrDqKV5Q9UdBNQC6+Fz9uSxjbNVWO8 DVKXwt3hhuD/XyWnI2JADvUgqveGjdZxpk3t7VNBbYtcPUCqg3BYDNA/VC+iDbAN J0DsFl+t7CATOXm52q/vRqZkVHXcq0Ubfp1zgzEcI4eDJKqtuqJ7Wq40qLIgVL7a znCULZL3UgHaAPrbGOhN4w3SzR5Nhvzwy8RrbzewF04S9BL6XR7jiK1WAcFtjm+H ZxcPViEXzl3ud1ZGR03ZyAHx2JclOGkTj4WCoTOk+AfkMLjccxZT40+Qsxj9YSh7 pZk4e5VxrstDZ8Dy8ZEeabPcwa0Pl5+F1ypJNj06Gki2ECVH5/2gwh804GYsmvpK wgSX2hIUW3euVsept741lysN1devUG8J/7Wtvtt34/yLsy1VMsTCO9DRqYBsyWeJ FsYvfV01MGUgE89YN3hlZrAxjk4VDpK25hswzsj7rpq0Qbs/1sfXY8+25mTvtt88 Ce3tyyJ5dIAIBQINY8bQpJfNXAN4GMMdo9TEmauK3zKYrxfyDpcoJLTPF8hZxluS Z3P4y+7W0uMrMYmn0CQg =BceG -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool
Hello Christopher,
Am 2013-05-07 17:20, schrieb Christopher Schultz:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
Michael,
On 5/7/13 6:43 AM, Michael-O wrote:
Von: Mark Thomas ma...@apache.org On 07/05/2013 10:25,
Michael-O wrote:
Von: Mark Thomas ma...@apache.org On 07/05/2013 09:16,
Michael-O wrote:
Hi folks,
I recently enabled a QueryTimeoutInterceptor with
queryTimeout of 60 seconds in a JDBC Pool data source
(7.0.37). When the app was shut down, Tomcat said: The web
application [/...] appears to have started a thread named
[OracleTimeoutPollingThread] but has failed to stop it...
We are using Oracle 11.2g with 11.2.0.3 JDBC drivers. I
have figured out that this thread is spawned by the driver
itself. According to this Stackoverflow answer [1] this is
a long-living thread, same says the JDBC FAQ [2] of
Oracle.
The thread seems to work like Pool's PoolCleaner thread. A
few month a ago I reported the same issue with the
PoolCleaner thread and Filip fixed the class loader
orders.
Can this be a false-positive by the memory leak detector
since this thread lives only once in the entire VM?
No. It is a memory leak and either or bug in your application
or in the JDBC driver.
Where is the Oracle JDBC driver? CATALINA_[BASE|HOME]/lib or
WEB-INF/lib
The driver is in the $CATALINA_HOME/lib only where
$CATALINA_BASE != $CATALINA_HOME. This was done for a single
webapp for testing purposes.
Does this make a difference?
The important thing is that it isn't in WEB-INF/lib.
How do you know that this is not a false-positive?
Experience, a lot of research into memory leaks and I wrote
Tomcat's memory leak detection code.
If you really know for sure, I can open a service request with
Oracle Support.
Good luck with that.
The problem is that when the driver creates the thread it does so
when the current class loader is the web application class
loader. That means that the Thread will be created with a context
class loader set to the web application class loader. That will
trigger a memory leak when the web application is stopped because
a reference is retained to the web application's class loader.
What the driver should do is, after it creates the thread, set
the thread's context class loader to the class loader that loaded
the driver.
What you are seeing is a variation of the leak described on page
15 of [1].
After reading the slides and your explanation this makes sense.
It's the same issue as Pool cleaner thread should be created using
the classloader that loaded the pool, not the context loader
(fhanik) fixed in 7.0.27.
I will file SR and let you know.
Note that you might be able to write your own code to mitigate this
problem, depending on exactly when that thread is created. If the
timeout thread isn't created until you actually try to issue a query
with a timeout, try something like this in a ServletContextListener's
contextInitialized method:
// NOTE: No resource management is being done in this example
// Get the current ClassLoader -- should be WebappClassLoader
ClassLoader cl = Thread.currentThread().getContextClassLoader();
// WebappClassLoader.getParent should be common loader
Thread.currentThread().setContextClassLoader(cl.getParent());
try
{
Connection conn = ...; // However you get a connection
Statement s = conn.createStatement();
s.setQueryTimeout(5000); // Doesn't really matter what TO is
s.execute(SELECT 1 FROM dual, );
}
finally
{
// Pop back to the original ClassLoader
Thread.currentThread().setContextClassLoader(cl);
}
This is the exact technique that Tomcat's
JreMemoryLeakPreventionListener uses to prevent ClassLoader leaks.
Yes, this looks like the way JDBC Pool does with the Pool Cleaner. I
would go for this as a last resort.
A
couple of notes:
1. This won't work under a SecurityManager. If you need to operate
under a SecurityManager, have a look at the
JreMemoryLeakPreventionListener code and adapt it to the above.
2. If the Oracle driver launches the thread when the DataSource is
created, it might happen too early for a ServletContextListener to
intervene. In that case, simply modify the
JreMEmoryLeakPreventionListener code directly. Patches are always welcome.
Not, it is not. I have attached VisualVM to Tomcat VM and have seen that
the thread is forked when the first query is executed.
3. If Oracle fixes this bug, Tomcat should not prevent against it. If
Oracle refuses to acknowledge/fix/etc. this bug, then it may make
sense to include such code in JreMemoryLeakPreventionListener, but it
should probably be done in such a way that a) it's not enabled by
default and b) the query used for triggering the Thread to be created
is user-selectable with maybe a reasonable default (like SELECT 1
FROM dual, as that tends to be valid in most RDBMSs).
I am already in contact with an Oracle engineer who has received a demo
WAR file to
Re: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
Michael,
On 5/7/13 4:33 PM, Michael-O wrote:
Am 2013-05-07 17:20, schrieb Christopher Schultz:
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256
Michael,
On 5/7/13 6:43 AM, Michael-O wrote:
Von: Mark Thomas ma...@apache.org On 07/05/2013 10:25,
Michael-O wrote:
Von: Mark Thomas ma...@apache.org On 07/05/2013
09:16, Michael-O wrote:
Hi folks,
I recently enabled a QueryTimeoutInterceptor with
queryTimeout of 60 seconds in a JDBC Pool data source
(7.0.37). When the app was shut down, Tomcat said: The
web application [/...] appears to have started a thread
named [OracleTimeoutPollingThread] but has failed to
stop it...
We are using Oracle 11.2g with 11.2.0.3 JDBC drivers.
I have figured out that this thread is spawned by the
driver itself. According to this Stackoverflow answer
[1] this is a long-living thread, same says the JDBC
FAQ [2] of Oracle.
The thread seems to work like Pool's PoolCleaner
thread. A few month a ago I reported the same issue
with the PoolCleaner thread and Filip fixed the class
loader orders.
Can this be a false-positive by the memory leak
detector since this thread lives only once in the
entire VM?
No. It is a memory leak and either or bug in your
application or in the JDBC driver.
Where is the Oracle JDBC driver? CATALINA_[BASE|HOME]/lib
or WEB-INF/lib
The driver is in the $CATALINA_HOME/lib only where
$CATALINA_BASE != $CATALINA_HOME. This was done for a
single webapp for testing purposes.
Does this make a difference?
The important thing is that it isn't in WEB-INF/lib.
How do you know that this is not a false-positive?
Experience, a lot of research into memory leaks and I wrote
Tomcat's memory leak detection code.
If you really know for sure, I can open a service request
with Oracle Support.
Good luck with that.
The problem is that when the driver creates the thread it
does so when the current class loader is the web application
class loader. That means that the Thread will be created with
a context class loader set to the web application class
loader. That will trigger a memory leak when the web
application is stopped because a reference is retained to the
web application's class loader.
What the driver should do is, after it creates the thread,
set the thread's context class loader to the class loader
that loaded the driver.
What you are seeing is a variation of the leak described on
page 15 of [1].
After reading the slides and your explanation this makes
sense. It's the same issue as Pool cleaner thread should be
created using the classloader that loaded the pool, not the
context loader (fhanik) fixed in 7.0.27.
I will file SR and let you know.
Note that you might be able to write your own code to mitigate
this problem, depending on exactly when that thread is created.
If the timeout thread isn't created until you actually try to
issue a query with a timeout, try something like this in a
ServletContextListener's contextInitialized method:
// NOTE: No resource management is being done in this example
// Get the current ClassLoader -- should be WebappClassLoader
ClassLoader cl = Thread.currentThread().getContextClassLoader();
// WebappClassLoader.getParent should be common loader
Thread.currentThread().setContextClassLoader(cl.getParent());
try { Connection conn = ...; // However you get a connection
Statement s = conn.createStatement(); s.setQueryTimeout(5000); //
Doesn't really matter what TO is s.execute(SELECT 1 FROM dual,
); } finally { // Pop back to the original ClassLoader
Thread.currentThread().setContextClassLoader(cl); }
This is the exact technique that Tomcat's
JreMemoryLeakPreventionListener uses to prevent ClassLoader
leaks.
Yes, this looks like the way JDBC Pool does with the Pool Cleaner.
I would go for this as a last resort.
A couple of notes:
1. This won't work under a SecurityManager. If you need to
operate under a SecurityManager, have a look at the
JreMemoryLeakPreventionListener code and adapt it to the above.
2. If the Oracle driver launches the thread when the DataSource
is created, it might happen too early for a
ServletContextListener to intervene. In that case, simply modify
the JreMEmoryLeakPreventionListener code directly. Patches are
always welcome.
Not, it is not. I have attached VisualVM to Tomcat VM and have seen
that the thread is forked when the first query is executed.
That's good news: it means that if you need it fixed *right away*, you
don't need to hack-and-recompile Tomcat: you can just do it in your
own ServletContextListener.
3. If Oracle fixes this bug, Tomcat should not prevent against
it. If Oracle refuses to acknowledge/fix/etc. this bug, then it
may make sense to include such code in
JreMemoryLeakPreventionListener, but it should probably be done
in such a way that a) it's not enabled by default and b) the
query
