Re: Database connections grow after redeploy
try netbeans its free and the profiler will watch all your objects being created. it will also work with snapshots... On Thu, 2008-03-13 at 20:56 +0200, Juha Laiho wrote: Scott McClanahan wrote: On Thu, 2008-03-13 at 12:02 -0400, Christopher Schultz wrote: Caldarale, Charles R wrote: | From: Scott McClanahan [mailto:[EMAIL PROTECTED] | Subject: Re: Database connections grow after redeploy | As far as the continuing data base connections, I can only speculate | that the prior instances of the webapp are still active, thereby | preventing cleanup of their resources. I agree. Scott: are you able to run your application through a tool that allows you to observe the heap and object graphs? More likely is that one or more application-level objects has retained a reference to a specific Connection object, which, of course, retains references back to the connection pool that created it. I would check any ServletContextListener classes you have, and then a generic search for putting things into the ServletContext, since that is one of the only places that applications typically store long-lived objects. Do you have any suggestions for a profiling tool like you described? Preferably open source. Thanks. Don't know about open source products, but I can vouch for YourKit Java Profiler. As far as I know, the main difference between YourKit and other profilers is that YourKit allows you to work through snapshots, whereas other profilers inject their probes into Java object creation methods. This means that more or less the only moment when YourKit has an effect on the performance of your application is when you take a memory snapshot. The other profilers I've seen attempt to trace each object allocation and deallocation in real time, which can be rather CPU consuming (especially if you're tracing a problem you cannot replicate in test environments). With YourKit, you take snapshots of the Java VM memory of your application, and compare them off-line (i.e. without needing any connection to the live application). - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Database connections grow after redeploy
of connections, number of busy connections and configuration details Display of system information including System.properties, memory usage bar and OS details Ability to show information about log files and download selected files Ability to tail log files in real time from a browser. Ability to interrupt execution of hang requests without server restart Support for DBCP, C3P0 and Oracle datasources Support for Tomcat 5.0.x and 5.5.x Support for Java 1.4 and Java 1.5 And here are some more that you can explore Tools for tracking down memory leaks. XPCOM Memory Leak tools (this document also has information on Mozilla's other memory leak tools). See also a tutorial on using these tools and the refcount balancer. Rational Purify®. (Commercial) GlowCode. (Commercial) ElectricFence. Electric Fence ... uses the virtual memory hardware of your system to detect when software overruns the boundaries of a malloc() buffer. It will also detect any accesses of memory that has been released by free(). Because it uses the VM hardware for detection, Electric Fence stops your program on the first instruction that causes a bounds violation. Specifically, it works by placing each heap allocation on its own VM page. Since the dynamic footprint of the browser is pretty large, this may make it tough to use in real life. Leaky. This is a home-grown tool that Kipp put together: Its a tool called leaky. What it does it help you track down memory leaks and some kinds of memory corruption. It also has entry pointers for logging addref/release calls. The key thing that leaky does is this: it logs all calls to malloc/free/realloc/new/delete into a log file. The logging data includes information about size and address, as well as the *call stack of the operation*. The leaky program then can translate the call stack data from addresses into symbols and then dump the data out. * refcount balancer. This is another home-grown tool that does finer-grained instrumentation and allows you to focus on specific objects rather than entire runs. Insure++®. (Commercial) Evaluation copy is free. Linux version. LeakTracer. ccmalloc. debauch Dead? memwatch. Geodesic Systems' GreatCircle Dead? Scott Furman has done some preliminary work to get this product working with Mozilla. Julian Seward writes of Valgrind, an open-source memory debugger and profiler for Linux/x86: `Snapshots 20020511 and later of the tool are able to run Mozilla correctly on both Red Hat 6.2 and 7.2. It will also do detailed low-level I1/D1/L2 cache profiling, so you can see which bits of Mozilla create cache misses. You'll need the patch from bug 124335 if working with versions prior to 1.0RC2. Profiling The following tools are used to measure product performance. Rational PurifyPlus. (Commercial) Used to be called Rational Quantify. qfy2html.pl. A Perl script for munging Quantify 6.0 output into hyperlinked HTML to share with your friends. Run Quantify, save your results as text, and then let this thing rip. Quantifying Mozilla on Solaris. Instructions on how to make Quantify play nicely with Mozilla on Solaris. For performance work on Mac, you can use either the Metrowerks Profiler (on subsets of the codebase), or Apple's Instrumentation SDK. For details, see the page on Mac performance. hiprof. A hierarchical instruction profiler for Digital Unix. There is an online tutorial, courtesy Jim Nance ([EMAIL PROTECTED]): I like it because it does not require you to recompile the program, because it give accurate times for callers, and because it comes with DU so that everyone developing there should already have it. * jprof. Sampling profiling tool by Jim Nance that runs on RedHat 6.1, 6.2. MOZ_TIMELINE Timing instrumentation. Extremely useful for app startup measurements. eazel profilers. Two profiling tools from Eazel/gnome.org, runs on Linux. JS Runtime Profiler. Profile JavaScript usage, dumps output to a file. Sysinternals utilities. Win9x/Me/NT/2K utilites for monitoring system usage. CPU/Mon, Diskmon, Filemon, Regmon, etc. gprof. Part of the GNU Binutils. Back in May 1999 there were rumors that gprof had trouble with dynamically linked and loaded objects (like Mozilla). Currently (March 2006) gprof is still being developed, so hopefully that's no longer true (assuming it ever was). The link to the gprof manual above is years out of date but, at the time of writing, it's the latest version of the manual on gnu.org. To get an up to date version of the manual, get the gprof source and run |. configure; make html| in the gprof source directory. - Original Message From: David Cassidy [EMAIL PROTECTED] To: Tomcat Users List users@tomcat.apache.org Sent: Friday, March 14, 2008 7:13:17 PM Subject: Re: Database connections grow after redeploy try netbeans its free and the profiler will watch all your objects being created. it will also work with snapshots... On Thu, 2008-03-13 at 20:56
RE: Database connections grow after redeploy
On Thu, 2008-03-13 at 09:23 -0500, Caldarale, Charles R wrote: From: Scott McClanahan [mailto:[EMAIL PROTECTED] Subject: Re: Database connections grow after redeploy Some times our exchange server really mangles e-mails or out right blocks them so this is just a resend in case it didn't make it through. Before resending, look in the archives to see if your original message is there (it is): http://marc.info/?l=tomcat-user As far as the continuing data base connections, I can only speculate that the prior instances of the webapp are still active, thereby preventing cleanup of their resources. There are numerous possible causes, including such things as spawning extra threads and not terminating them when the webapp is told to shut down. You should be able to work around the problem by making the connection pool global rather than local to the webapp. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] Making the DBCP a global resource and linking it into the applications specifically does fix the problem. Good suggestion. Thanks. - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Database connections grow after redeploy
On Fri, 2008-03-14 at 13:43 +, David Cassidy wrote: try netbeans its free and the profiler will watch all your objects being created. it will also work with snapshots... On Thu, 2008-03-13 at 20:56 +0200, Juha Laiho wrote: Scott McClanahan wrote: On Thu, 2008-03-13 at 12:02 -0400, Christopher Schultz wrote: Caldarale, Charles R wrote: | From: Scott McClanahan [mailto:[EMAIL PROTECTED] | Subject: Re: Database connections grow after redeploy | As far as the continuing data base connections, I can only speculate | that the prior instances of the webapp are still active, thereby | preventing cleanup of their resources. I agree. Scott: are you able to run your application through a tool that allows you to observe the heap and object graphs? More likely is that one or more application-level objects has retained a reference to a specific Connection object, which, of course, retains references back to the connection pool that created it. I would check any ServletContextListener classes you have, and then a generic search for putting things into the ServletContext, since that is one of the only places that applications typically store long-lived objects. Do you have any suggestions for a profiling tool like you described? Preferably open source. Thanks. Don't know about open source products, but I can vouch for YourKit Java Profiler. As far as I know, the main difference between YourKit and other profilers is that YourKit allows you to work through snapshots, whereas other profilers inject their probes into Java object creation methods. This means that more or less the only moment when YourKit has an effect on the performance of your application is when you take a memory snapshot. The other profilers I've seen attempt to trace each object allocation and deallocation in real time, which can be rather CPU consuming (especially if you're tracing a problem you cannot replicate in test environments). With YourKit, you take snapshots of the Java VM memory of your application, and compare them off-line (i.e. without needing any connection to the live application). - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] Thanks for all of the suggestions! - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Database connections grow after redeploy
of connections, number of busy connections and configuration details Display of system information including System.properties, memory usage bar and OS details Ability to show information about log files and download selected files Ability to tail log files in real time from a browser. Ability to interrupt execution of hang requests without server restart Support for DBCP, C3P0 and Oracle datasources Support for Tomcat 5.0.x and 5.5.x Support for Java 1.4 and Java 1.5 And here are some more that you can explore Tools for tracking down memory leaks. XPCOM Memory Leak tools (this document also has information on Mozilla's other memory leak tools). See also a tutorial on using these tools and the refcount balancer. Rational Purify®. (Commercial) GlowCode. (Commercial) ElectricFence. Electric Fence ... uses the virtual memory hardware of your system to detect when software overruns the boundaries of a malloc() buffer. It will also detect any accesses of memory that has been released by free(). Because it uses the VM hardware for detection, Electric Fence stops your program on the first instruction that causes a bounds violation. Specifically, it works by placing each heap allocation on its own VM page. Since the dynamic footprint of the browser is pretty large, this may make it tough to use in real life. Leaky. This is a home-grown tool that Kipp put together: Its a tool called leaky. What it does it help you track down memory leaks and some kinds of memory corruption. It also has entry pointers for logging addref/release calls. The key thing that leaky does is this: it logs all calls to malloc/free/realloc/new/delete into a log file. The logging data includes information about size and address, as well as the *call stack of the operation*. The leaky program then can translate the call stack data from addresses into symbols and then dump the data out. * refcount balancer. This is another home-grown tool that does finer-grained instrumentation and allows you to focus on specific objects rather than entire runs. Insure++®. (Commercial) Evaluation copy is free. Linux version. LeakTracer. ccmalloc. debauch Dead? memwatch. Geodesic Systems' GreatCircle Dead? Scott Furman has done some preliminary work to get this product working with Mozilla. Julian Seward writes of Valgrind, an open-source memory debugger and profiler for Linux/x86: `Snapshots 20020511 and later of the tool are able to run Mozilla correctly on both Red Hat 6.2 and 7.2. It will also do detailed low-level I1/D1/L2 cache profiling, so you can see which bits of Mozilla create cache misses. You'll need the patch from bug 124335 if working with versions prior to 1.0RC2. Profiling The following tools are used to measure product performance. Rational PurifyPlus. (Commercial) Used to be called Rational Quantify. qfy2html.pl. A Perl script for munging Quantify 6.0 output into hyperlinked HTML to share with your friends. Run Quantify, save your results as text, and then let this thing rip. Quantifying Mozilla on Solaris. Instructions on how to make Quantify play nicely with Mozilla on Solaris. For performance work on Mac, you can use either the Metrowerks Profiler (on subsets of the codebase), or Apple's Instrumentation SDK. For details, see the page on Mac performance. hiprof. A hierarchical instruction profiler for Digital Unix. There is an online tutorial, courtesy Jim Nance ([EMAIL PROTECTED]): I like it because it does not require you to recompile the program, because it give accurate times for callers, and because it comes with DU so that everyone developing there should already have it. * jprof. Sampling profiling tool by Jim Nance that runs on RedHat 6.1, 6.2. MOZ_TIMELINE Timing instrumentation. Extremely useful for app startup measurements. eazel profilers. Two profiling tools from Eazel/gnome.org, runs on Linux. JS Runtime Profiler. Profile JavaScript usage, dumps output to a file. Sysinternals utilities. Win9x/Me/NT/2K utilites for monitoring system usage. CPU/Mon, Diskmon, Filemon, Regmon, etc. gprof. Part of the GNU Binutils. Back in May 1999 there were rumors that gprof had trouble with dynamically linked and loaded objects (like Mozilla). Currently (March 2006) gprof is still being developed, so hopefully that's no longer true (assuming it ever was). The link to the gprof manual above is years out of date but, at the time of writing, it's the latest version of the manual on gnu.org. To get an up to date version of the manual, get the gprof source and run |. configure; make html| in the gprof source directory. - Original Message From: David Cassidy [EMAIL PROTECTED] To: Tomcat Users List users@tomcat.apache.org Sent: Friday, March 14, 2008 7:13:17 PM Subject: Re: Database connections grow after redeploy try netbeans its free and the profiler will watch all your objects being created. it will also work with snapshots... On Thu, 2008-03-13 at 20:56
RE: Database connections grow after redeploy
From: Scott McClanahan [mailto:[EMAIL PROTECTED] Subject: RE: Database connections grow after redeploy Making the DBCP a global resource and linking it into the applications specifically does fix the problem. Well, really it just masks the problem. Classes and objects from the replaced webapp are likely still around, eating up heap space, especially PermGen. That will cause OOME problems after some number of redeployments. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Database connections grow after redeploy
On Wed, 2008-03-12 at 14:31 -0400, Scott McClanahan wrote: We are using the dbcp capabilities of tomcat and configured the datasource in the applications context configuration file. In a few simple tests we've seen some unexpected behavior and aren't sure if it's an error on our part. We are running tomcat 5.5.26 with the compatibility libraries on java 1.4.2_16. Also, CentOS 5.1. Basically, we setup the datasource to have an initialSize=5, maxIdle=10, minIdle=5. We made a simple request to the web application and immediately 5 connections to database were created (as expected). When we redeployed the application lambda probe reported zero connections to the database in that datasource (as expected). Netstat on the server reported 5 connections still and tcpdump showed that the validation queries were still being done across those connections. The moment we made the same simple request to the application again 5 new connections were created and the 5 previous connections were not used at all for the database queries. If we follow this same procedure any arbitrary number of times 5 additional connections are created and the previous 5 are useless (although still validated with the validationquery). Is this expected behavior during a redeploy or application restart? We are using the connection pooling libraries bundled in tomcat and haven't loaded any pooling libraries in the application. Thanks. Some times our exchange server really mangles e-mails or out right blocks them so this is just a resend in case it didn't make it through. Anybody else seeing this behavior? More details can be made available. - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Database connections grow after redeploy
From: Scott McClanahan [mailto:[EMAIL PROTECTED] Subject: Re: Database connections grow after redeploy Some times our exchange server really mangles e-mails or out right blocks them so this is just a resend in case it didn't make it through. Before resending, look in the archives to see if your original message is there (it is): http://marc.info/?l=tomcat-user As far as the continuing data base connections, I can only speculate that the prior instances of the webapp are still active, thereby preventing cleanup of their resources. There are numerous possible causes, including such things as spawning extra threads and not terminating them when the webapp is told to shut down. You should be able to work around the problem by making the connection pool global rather than local to the webapp. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Database connections grow after redeploy
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Chuck, Caldarale, Charles R wrote: | From: Scott McClanahan [mailto:[EMAIL PROTECTED] | Subject: Re: Database connections grow after redeploy | | As far as the continuing data base connections, I can only speculate | that the prior instances of the webapp are still active, thereby | preventing cleanup of their resources. I agree. Scott: are you able to run your application through a tool that allows you to observe the heap and object graphs? Many profilers will allow you to do this. You could look for instances of the connection pool and see what is holding onto them. It's unusual for an application to hold references to the pool, but not completely impossible. More likely is that one or more application-level objects has retained a reference to a specific Connection object, which, of course, retains references back to the connection pool that created it. I would check any ServletContextListener classes you have, and then a generic search for putting things into the ServletContext, since that is one of the only places that applications typically store long-lived objects. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.8 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkfZUDAACgkQ9CaO5/Lv0PBwCwCfbz4bORVnNboqnMWZu6GQClbA CNAAn2BIaoZChNHbsks7/tLdK75S2A3X =4Ssq -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Database connections grow after redeploy
On Thu, 2008-03-13 at 12:02 -0400, Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Chuck, Caldarale, Charles R wrote: | From: Scott McClanahan [mailto:[EMAIL PROTECTED] | Subject: Re: Database connections grow after redeploy | | As far as the continuing data base connections, I can only speculate | that the prior instances of the webapp are still active, thereby | preventing cleanup of their resources. I agree. Scott: are you able to run your application through a tool that allows you to observe the heap and object graphs? Many profilers will allow you to do this. You could look for instances of the connection pool and see what is holding onto them. It's unusual for an application to hold references to the pool, but not completely impossible. More likely is that one or more application-level objects has retained a reference to a specific Connection object, which, of course, retains references back to the connection pool that created it. I would check any ServletContextListener classes you have, and then a generic search for putting things into the ServletContext, since that is one of the only places that applications typically store long-lived objects. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.8 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkfZUDAACgkQ9CaO5/Lv0PBwCwCfbz4bORVnNboqnMWZu6GQClbA CNAAn2BIaoZChNHbsks7/tLdK75S2A3X =4Ssq -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] Do you have any suggestions for a profiling tool like you described? Preferably open source. Thanks. - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Database connections grow after redeploy
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Scott, Scott McClanahan wrote: | Do you have any suggestions for a profiling tool like you described? | Preferably open source. Thanks. I have used Borland's OptimizeIt in the past, but I think it's a defunct product (and it's also neither free nor open source). I don't have experience with other profilers or debuggers, but lots of folks on the list have mentioned JProfiler (http://www.ej-technologies.com/products/jprofiler/overview.html) which has a free trial, but I don't believe it is free. My understanding is that it is a very good tool, though. JProbe is also brought up on the list occasionally. Also not free, also has a free trial. Search for archives of this list for discussions concerning profiles, debuggers and such, because I apparently don't have any good recommendations ;) Good luck, - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.8 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkfZaUsACgkQ9CaO5/Lv0PBNVQCgvDLEJ66KelhIqsmuMCae1pkB 6uQAnif2EQ35yv5I2KtUrTPMxDDS8PTE =7OQZ -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Database connections grow after redeploy
From: Scott McClanahan [mailto:[EMAIL PROTECTED] Subject: Re: Database connections grow after redeploy Do you have any suggestions for a profiling tool like you described? Just to get started, the Sun JDK comes with a primitive heap profiler. You can enable it with the command line option: -agentlib:hprof=heap=sites Documentation is here (among other places): http://java.sun.com/developer/technicalArticles/Programming/HPROF.html - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Database connections grow after redeploy
Scott McClanahan wrote: On Thu, 2008-03-13 at 12:02 -0400, Christopher Schultz wrote: Caldarale, Charles R wrote: | From: Scott McClanahan [mailto:[EMAIL PROTECTED] | Subject: Re: Database connections grow after redeploy | As far as the continuing data base connections, I can only speculate | that the prior instances of the webapp are still active, thereby | preventing cleanup of their resources. I agree. Scott: are you able to run your application through a tool that allows you to observe the heap and object graphs? More likely is that one or more application-level objects has retained a reference to a specific Connection object, which, of course, retains references back to the connection pool that created it. I would check any ServletContextListener classes you have, and then a generic search for putting things into the ServletContext, since that is one of the only places that applications typically store long-lived objects. Do you have any suggestions for a profiling tool like you described? Preferably open source. Thanks. Don't know about open source products, but I can vouch for YourKit Java Profiler. As far as I know, the main difference between YourKit and other profilers is that YourKit allows you to work through snapshots, whereas other profilers inject their probes into Java object creation methods. This means that more or less the only moment when YourKit has an effect on the performance of your application is when you take a memory snapshot. The other profilers I've seen attempt to trace each object allocation and deallocation in real time, which can be rather CPU consuming (especially if you're tracing a problem you cannot replicate in test environments). With YourKit, you take snapshots of the Java VM memory of your application, and compare them off-line (i.e. without needing any connection to the live application). -- ..Juha - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
