Re: Would like to monitor memory use offline
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Richard, Richard S. Huntrods wrote: | Everything is local except the pooled connections, so I would say this | is the problem. This code was originally written before tomcat had good | connection pools, and so I had to write my own. The pool is basically a | vector of connections. I wouldn't want to blame your connection pool just yet; I'm just saying that it is a possibility. The open-source connection pools have a /lot/ of users and a /lot/ of eyes looking at them, and tend to be very stable. Your in-house connection pool may not have been as thoroughly tested. | At a more fundamental level, I still don't think the new connector is | working correctly (and I've read many of the forum discussions about | this...). Has anyone mentioned the behavior you are observing? Were those posts anything but idle speculation, or has anyone come to an actual conclusion about the cause of the errors? | If I close the resultset AND the statement, then the connector | should release all the objects created by those two. The connection is, | after all, just a pipe between the database and the java code. The | connector should not (IMO) be hanging on to statement or resultset | objects just because the connection is still in existance. I completely agree. Two things that I can think of that might be causing problems with the Connector itself: 1. ResultSetMetadata -- use of the metadata methods can cause additional queries to be executed, which means more ResultSet objects, Fields, etc. I didn't see any use of this in your sample code, so I suspect this is not the issue. 2. Client-side PreparedStatement caching. Have you enabled any of the caching options provided by the driver? I'm guessing not, since you were using a 3.x driver before, which IIRC does not support such caching. | Still - this is most clearly NOT my problem as I can verify my code is | working normally (i.e. no exceptions happening) and thus properly | closing both the resultset and the statement. Agreed. | For fun I put try-catch blocks with additional resultset and statemtent | calls to close in the finally block and it made no difference to the | memory leak. Right. I would not have predicted that you would change anything by doing this -- it's just a good practice to get into in the event that exceptions /do/ start cropping up. | Yea - the code was developed a long while ago. There's also a single | version that can run under Tomcat or as offline processes and I don't | want to have to write two different database mechanisms to support the | application outside of Tomcat. Tomcat uses JNDI DataSources to provide connection pooling. This is very easy to do outside of Tomcat if you are interested. Another option would be gut your existing CP implementation and replace it with pass-through calls to a library like commons-dbcp. This is exactly what I did with a project that I inherited a few years ago and many, many JDBC-related problems went away (more to do with connection leakage than memory leakage). My last suggestion would be to use a real memory profiler. These tools can typically tell you exactly where in your code a particular object was instantiated. After you run your program for a while, inspect the heap (and all those Field objects) and see where they are being created. You may find that you are looking at fixing the wrong method in your own code -- or that the driver itself is leaking Field objects (which would be unlikely, but certainly possible). Good luck, - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkihmQgACgkQ9CaO5/Lv0PCT+gCfcnyqkmwMAyiCZUlsCr0QjmIV /0sAnj5FOECT7QlfIad7PMX8xg0Tc6ud =HbMZ -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Would like to monitor memory use offline
Richard, Richard S. Huntrods wrote: | Everything is local except the pooled connections, so I would say this | is the problem. This code was originally written before tomcat had good | connection pools, and so I had to write my own. The pool is basically a | vector of connections. I wouldn't want to blame your connection pool just yet; I'm just saying that it is a possibility. The open-source connection pools have a /lot/ of users and a /lot/ of eyes looking at them, and tend to be very stable. Your in-house connection pool may not have been as thoroughly tested. My in-house connection pool was based on code examples from around 2001 - when I first implemented the system. My database classes were from the same time, hence the almost correct nature of the code (close statements, finally clause). It all worked without incident until this July, when I was required to upgrade an aging server and also upgraded all of the packages. We had been using the original mmjb.jar version (predating when MySQL took over the connector/J project). It took a while to trace the memory leaks to the connector, and then it was quick work to find the version split where the leak started. (see prior posts). In that time I had been reading as much as possible about Tomcat memory use, leaks, and finally stuff about the connector. | At a more fundamental level, I still don't think the new connector is | working correctly (and I've read many of the forum discussions about | this...). Has anyone mentioned the behavior you are observing? Were those posts anything but idle speculation, or has anyone come to an actual conclusion about the cause of the errors? The majority of the posts I'm referring to occurred in 2004. Many people were noticing memory leaks with the connectors, and wanting to call them Bugs. The database folks and connector authors fiercly defended their work and stated the problem was always due to missing close() statements. The discussion then turned toward misinterpretation of the JDBC specification - where the people complaining about the leak insisted the spec. required the connector to release all memory objects as soon as the statement went out of scope and the connector people saying No - you must call close(). SO I put the appropriate close() for statement in my code - but to no effect. With the exception of my connection, my code now matches what is correct. | If I close the resultset AND the statement, then the connector | should release all the objects created by those two. The connection is, | after all, just a pipe between the database and the java code. The | connector should not (IMO) be hanging on to statement or resultset | objects just because the connection is still in existance. I completely agree. Two things that I can think of that might be causing problems with the Connector itself: 1. ResultSetMetadata -- use of the metadata methods can cause additional queries to be executed, which means more ResultSet objects, Fields, etc. I didn't see any use of this in your sample code, so I suspect this is not the issue. Actually, it is. I use a RSMD object in all my queries. I don't close it. - MAYBE THAT's THE PROBLEM. I will test and report... 2. Client-side PreparedStatement caching. Have you enabled any of the caching options provided by the driver? I'm guessing not, since you were using a 3.x driver before, which IIRC does not support such caching. Nope. No caching at all. | Still - this is most clearly NOT my problem as I can verify my code is | working normally (i.e. no exceptions happening) and thus properly | closing both the resultset and the statement. Agreed. | For fun I put try-catch blocks with additional resultset and statemtent | calls to close in the finally block and it made no difference to the | memory leak. Right. I would not have predicted that you would change anything by doing this -- it's just a good practice to get into in the event that exceptions /do/ start cropping up. Agreed. | Yea - the code was developed a long while ago. There's also a single | version that can run under Tomcat or as offline processes and I don't | want to have to write two different database mechanisms to support the | application outside of Tomcat. Tomcat uses JNDI DataSources to provide connection pooling. This is very easy to do outside of Tomcat if you are interested. Another option would be gut your existing CP implementation and replace it with pass-through calls to a library like commons-dbcp. This is exactly what I did with a project that I inherited a few years ago and many, many JDBC-related problems went away (more to do with connection leakage than memory leakage). Sounds like a plan - if ever I have time to do this. ;-) My last suggestion would be to use a real memory profiler. These tools can typically tell you exactly where in your code a particular object was instantiated. After you run your program for a while, inspect the heap (and
Re: Would like to monitor memory use offline
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Richard, Richard S. Huntrods wrote: | I use a RSMD object in all my queries. I don't close | it. - MAYBE THAT's THE PROBLEM. I will test and report... Looking back at the JDBC API, I'm not sure you have any control over the RSMD query, result, etc. Sorry for the red herring (if it is one)! I think he memory profiler is the way to go. Lots of folks on the list like JProbe and YourKit. I have no experience with either of them. The last profiler I used was Borland's OptimizeIt a long time ago, and I was very happy with it. I'm sure that newer versions would be even better. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkihtSkACgkQ9CaO5/Lv0PAYrACeMnmzBLP0jaHBuC1hCE0qmq90 geQAnjVSupORdSjqScEttRD2sef8Mdng =LhUD -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Would like to monitor memory use offline
Chris,
| If I close the resultset AND the statement, then the connector
| should release all the objects created by those two. The connection is,
| after all, just a pipe between the database and the java code. The
| connector should not (IMO) be hanging on to statement or resultset
| objects just because the connection is still in existance.
I completely agree. Two things that I can think of that might be causing
problems with the Connector itself:
1. ResultSetMetadata -- use of the metadata methods can cause additional
queries to be executed, which means more ResultSet objects, Fields, etc.
I didn't see any use of this in your sample code, so I suspect this is
not the issue.
OK. I tried closing RSMD's. You can't close them so I just made them
null. It had no effect, but I'm not surprised since they come from the
ResultSet and should (in theory) be destroyed when the ResultSet is
closed (IMO).
HOWEVER - In messing with the code again ...
I FIXED THE PROBLEM!
Yep - totally fixed (tested and verified).
What I decided to do was to move the close statements (and nulling RSMD)
into the FINALLY block - it seemed pointless to duplicate code. When you
must have the close() statements in finally, why put them in the main
code as well?
So I modified ALL my DBMS methods as follows (just showing the finally
block):
finally {
try {
if(resultSet != null) {
resultSet.close();
resultSet = null;
}
if(statement != null) {
statement.close();
statement = null;
}
}
catch (Exception e) {
}
}
In doing this, I found several methods I missed earlier - they were
update and delete methods that didn't use ResultSet so I had missed
adding the statement.close().
So once I was done, EVERY METHOD in the DBMS class had appropriate
closes on resultsets and statements (as appropriate) - all placed in the
finally block.
Testing proved this to completely fix the memory leak.
Thanks again VERY much for all the advice and assistance!!!
:-)
Cheers,
-Richard
-
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Would like to monitor memory use offline
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Richard,
Richard S. Huntrods wrote:
| I FIXED THE PROBLEM!
|
| Yep - totally fixed (tested and verified).
|
| What I decided to do was to move the close statements (and nulling RSMD)
| into the FINALLY block - it seemed pointless to duplicate code. When you
| must have the close() statements in finally, why put them in the main
| code as well?
|
| So I modified ALL my DBMS methods as follows (just showing the finally
| block):
|
| finally {
| try {
| if(resultSet != null) {
| resultSet.close();
| resultSet = null;
| }
| if(statement != null) {
| statement.close();
| statement = null;
| }
| }
| catch (Exception e) {
| }
| }
|
| In doing this, I found several methods I missed earlier - they were
| update and delete methods that didn't use ResultSet so I had missed
| adding the statement.close().
I told you. ;)
Note that you need to have each close() in its own try/catch block: you
don't want an exception in closing the ResultSet object to prevent your
Statement from closing.
| So once I was done, EVERY METHOD in the DBMS class had appropriate
| closes on resultsets and statements (as appropriate) - all placed in the
| finally block.
Yup. There can always be that /one/ method that's not fixed, and if you
run it enough times, it'll kill you.
I suspect that the change you are experiencing between Connector/J
versions is that the newer versions either do not implement finalizers
(which were hardly implemented in certain JVMs, anyway) anymore, or some
sort of cleanup code was removed, because the JDBC spec was clarified to
mandate that resources be cleaned up by the calling code (and therefore
the driver is not responsible for sloppy coding of the client).
Glad you have your solution. Next time remember that you might be so
deep in your own code that you aren't open to questions and suggestions.
Remember to re-evaluate your assumptions, especially when someone is
trying to help ;)
Good luck,
- -chris
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEARECAAYFAkihx4oACgkQ9CaO5/Lv0PB8QwCgnhWLFBi3GfWLEpYCKfFjBRbT
9OkAnAobAbaQOFKVurnnnSZto7tUTiMJ
=F1Pl
-END PGP SIGNATURE-
-
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Would like to monitor memory use offline
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Richard,
Richard S. Huntrods wrote:
| Richard,
|
| Richard S. Huntrods wrote:
| |public static Vector listLookup(String table) {
| | //Connection connection = null; // connection is managed by a
| | connection pool
|
| So, is 'connection' a local or not?
| It's part of my code (I use my own connection pool class) but not to the
| DBMS code. Only if you can't get a connection from the pool
| (connection==null) do you create a local one for this method.
|
| If the connection from the pool is OK, you use it then release it back
| to the pool. If it's local (because you could not get one from the pool)
| then you close it in the finally block.
I think you are misunderstanding me: I'm asking if you are using a
shared java.sql.Connection reference among all threads running your db
code. If you are, then you've got a big problem, which could explain
some of the memory leaks you are seeing. If you have a class-level
reference like this:
public class MyDBMSMethods
{
~ private Connection connection;
~ public List getMyRecords()
~ {
~ connection = getConnectionFromPool();
~ ...
~ }
}
... then you should expect lots of problems. From your posted code, it
looks like this is what you are doing.
All connection references should be strictly local to each method. Each
method should get its own connection from the connection pool, use it,
and return it to the pool (unless you accept the connection as a
parameter to the method, in which case, you should use it).
| You need to have statement.close() and resultSet.close() in here (in the
| finally block), not up above. Also, most database connection pools
| require that you call connection.close() to return the connection to the
| pool. Are you ever calling connection.close()?
|
| Um, sorry to disagree, but that is not really correct and also does not
| touch my problem.
Not closing your resources in a finally block it surely /not/ correct,
and you /can/ leak resources in this way. Which part of my statement is
not correct?
| I am already closing the resultSet and statement in the normal
| execution part of my code. At no time during the execution of this code
| in these tests was an exception thrown in this method
It doesn't matter. If you do not close your resources in a finally
block, they are not guaranteed to be closed. It might not be your
current problem, but it could be your future problem.
| so *I was now
| correctly closing the statement and resultSet. YET IN SPITE OF THIS the
| versions of the connector after 3.1.10 FAIL to release the Field objects.
|
| The other reason NOT to put close() in the fianlly block is that the
| close() methods can throw and exception.
That's why you wrap the call to close() in its own try/catch block.
| It is very bad programming
| practice to put exception throwing code' in a finally block. Sure, you
| could use another try-catch structure for the close() statements, but
| really - you should have already closed the stuff in the normal
| execution (as I did).
If you say so. My code doesn't leak DB resources, and it is written in
this way.
| I did state one thing in error - the finally block (of course) ALWAYS
| gets called, so you only want code in that block that must be done after
| all else has happened, but you also don't really want to be repeating
| code that should go elsewhere (like the close()).
|
|
| Also, which connection pool do you use?
|
| My own.
Is there a compelling reason not to use one of the freely-available,
well-supported connection pools available? Tomcat even has one already
built into it.
| Yea - the problem may be in the connection pool that I wrote, but I
| still find it odd that closing the resultSet() and statement() has no
| effect using connectors from 3.1.10 and on. Connector 3.0.17b and prior
| work just fine and release the objects.
If that's the case, I would stick with the Connector/J version that works.
- -chris
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEARECAAYFAkigSLgACgkQ9CaO5/Lv0PCatgCgkjmgLpyS0ZgGHjr1mHnIWCXO
4kIAnjZHj+16aoc2DY8ScKJmBqP8Y4rx
=rQ35
-END PGP SIGNATURE-
-
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Would like to monitor memory use offline
Richard,
Richard S. Huntrods wrote:
| Richard,
|
| Richard S. Huntrods wrote:
| |public static Vector listLookup(String table) {
| | //Connection connection = null; // connection is managed
by a
| | connection pool
|
| So, is 'connection' a local or not?
| It's part of my code (I use my own connection pool class) but not to
the
| DBMS code. Only if you can't get a connection from the pool
| (connection==null) do you create a local one for this method.
|
| If the connection from the pool is OK, you use it then release it back
| to the pool. If it's local (because you could not get one from the
pool)
| then you close it in the finally block.
I think you are misunderstanding me: I'm asking if you are using a
shared java.sql.Connection reference among all threads running your db
code. If you are, then you've got a big problem, which could explain
some of the memory leaks you are seeing. If you have a class-level
reference like this:
public class MyDBMSMethods
{
~ private Connection connection;
~ public List getMyRecords()
~ {
~ connection = getConnectionFromPool();
~ ...
~ }
}
... then you should expect lots of problems. From your posted code, it
looks like this is what you are doing.
All connection references should be strictly local to each method. Each
method should get its own connection from the connection pool, use it,
and return it to the pool (unless you accept the connection as a
parameter to the method, in which case, you should use it).
Everything is local except the pooled connections, so I would say this
is the problem. This code was originally written before tomcat had good
connection pools, and so I had to write my own. The pool is basically a
vector of connections.
At a more fundamental level, I still don't think the new connector is
working correctly (and I've read many of the forum discussions about
this...). If I close the resultset AND the statement, then the connector
should release all the objects created by those two. The connection is,
after all, just a pipe between the database and the java code. The
connector should not (IMO) be hanging on to statement or resultset
objects just because the connection is still in existance.
| You need to have statement.close() and resultSet.close() in here
(in the
| finally block), not up above. Also, most database connection pools
| require that you call connection.close() to return the connection
to the
| pool. Are you ever calling connection.close()?
|
| Um, sorry to disagree, but that is not really correct and also does not
| touch my problem.
Not closing your resources in a finally block it surely /not/ correct,
and you /can/ leak resources in this way. Which part of my statement is
not correct?
Yea - looked further and I have seen try/catch blocks in finally code.
Still - this is most clearly NOT my problem as I can verify my code is
working normally (i.e. no exceptions happening) and thus properly
closing both the resultset and the statement.
For fun I put try-catch blocks with additional resultset and statemtent
calls to close in the finally block and it made no difference to the
memory leak.
| I am already closing the resultSet and statement in the normal
| execution part of my code. At no time during the execution of this
code
| in these tests was an exception thrown in this method
It doesn't matter. If you do not close your resources in a finally
block, they are not guaranteed to be closed. It might not be your
current problem, but it could be your future problem.
I will not disagree with this. That's why the finally is there in the
first place.
| so *I was now
| correctly closing the statement and resultSet. YET IN SPITE OF THIS the
| versions of the connector after 3.1.10 FAIL to release the Field
objects.
|
| The other reason NOT to put close() in the fianlly block is that the
| close() methods can throw and exception.
That's why you wrap the call to close() in its own try/catch block.
| It is very bad programming
| practice to put exception throwing code' in a finally block. Sure, you
| could use another try-catch structure for the close() statements, but
| really - you should have already closed the stuff in the normal
| execution (as I did).
If you say so. My code doesn't leak DB resources, and it is written in
this way.
| I did state one thing in error - the finally block (of course) ALWAYS
| gets called, so you only want code in that block that must be done
after
| all else has happened, but you also don't really want to be repeating
| code that should go elsewhere (like the close()).
|
|
| Also, which connection pool do you use?
|
| My own.
Is there a compelling reason not to use one of the freely-available,
well-supported connection pools available? Tomcat even has one already
built into it.
Yea - the code was developed a long while ago. There's also a single
version that can run under Tomcat or as offline processes and I don't
want to
Re: Would like to monitor memory use offline
Chris,
Richard,
Richard S. Huntrods wrote:
| In my code I was calling resultSet.close(), but not statement.close().
That'll do it.
Actually, fixing it did NOT help. See more below...
| The problem is, even though I verified (debug statements) that the call
| is being made, the memory is STILL not being released. If I run the
same
| large query 3 times in a row, my memory use triples. Even if I log off
| the session (my application) and invalidate the session, the memory is
| still locked up and cannot be freed by the JVM.
Can you post some (sanitized) code? Maybe you're still missing something.
My sanitized code looks exactly like the code you posted below. I put
all my actual database work inside a DBMS class so that I could change
it out anytime without messing the application, and it's worked well.
Each different type of database access has it's own method (lookup,
lookupList, insert, delete, update, etc...). Each method opens ONE
connection, ONE statement and gets ONE resultSet. When I'm done, I close
the resultSet and (now) the statement. I also have full exception
handlers plus the necessary finally clause.
I also know from running my application that no exceptions are occurring
during the memory leak (no output to the debug file).
Remember that you can't just close the statement when you're done with
your method... if you use multiple Statements and ResultSets, you have
to close them as you so. You also have to make sure that you have a
finally block that closes them in case of an exception condition.
I never use multiple statements and resultSets. That's just asking for
trouble, IMO. ;-)
I've posted it before, but I'll to it again because it bears repeating:
here is the proper way to write JDBC code:
Connection conn = null;
~PreparedStatement ps = null;
~ResultSet rs = null;
~try
~{
~conn = getConnection(); // however you get your connections
~ps = conn.prepareStatement(...); // your query
// set params, etc.
~rs = ps.executeQuery();
// do whatever you need to do with your ResultSet
~}
~finally
~{
if(null != rs) { rs.close(); } catch (SQLException sqle)
{ /* log this error! */ }
if(null != ps) { ps.close(); } catch (SQLException sqle)
{ /* log this error! */ }
if(null != conn) { conn.close(); } catch (SQLException sqle)
{ /* log this error! */ }
~}
Yep - my code looks pretty much just like this.
I wrote a method in a utility class that does everything in the finally
block -- it makes the code much easier to read IMO.
Agreed. Me too.
Note that if you need multiple Statements, you should either use them
serially, and close each statement before you move onto the next one
(and re-use the local variable), or you should define them all outside
of the try/finally block and make sure they are closed under all
conditions. If you don't do this, you'll leak memory.
If you are using transactions, remember that you must catch /every
exception that can possibly occur/ (yes, even Errors and
RuntimeExceptions) and make sure you do a rollback before you close
everything.
| So what am I missing? I was sure that adding the code to close my
| statements would fix the problem.
Is it possible that the code you're looking at is good, but some other
code has a similar leak but is yet to be fixed.
As I said, 3.0.17 works PERFECTLY (no memory leaks, memory use hovers
around 50 megs of stack. With 3.1.10 and newer, memory use immediately
starts to climb immediately. It quickly reaches (within 30 min) 500
megabtyes ( of a 1 gig stack) and after an hour it hits the stack limit
and tomcat crashes. This is even with the UseParallelOLDGC garbage
collector in use to compact the old stack (without it the system crashes
even sooner due to fragmenting).
| So - does anyone know what the major change was between 3.0.17 and
| 3.1.10 that would have such a dramatic effect?
http://dev.mysql.com/doc/refman/5.0/en/cj-news.html
It's possible that newer versions of the JDBC driver more properly
adhere to the JDBC specification which results in memory leakage
(because JDBC expects you to code in very rigid ways, including cleaning
up your own memory messes ;) .
That's what I am thinking as well - or a newer JDBC spec. Otherwise how
to explain a ONE DAY release date difference by a MAJOR version change
(3.0.17 on June 22/05 - 3.1.10 on June 23/05).
I use mysql-connector-5.0.8.jar and have not had a problem in a very
long time.
I am quite sure that something was changed in the spec that altered the
way the code should be written, and the old way (3.0.17) was probably
automatically cleaning up while the new way (3.1.10 and above) expects
the programmer to clean up.
PROBLEM IS, according to all docs I *am* doing the proper cleanup.
HERE IS MY neutered code:
public static Vector listLookup(String table) {
Re: Would like to monitor memory use offline
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Richard,
Richard S. Huntrods wrote:
|public static Vector listLookup(String table) {
| //Connection connection = null; // connection is managed by a
| connection pool
So, is 'connection' a local or not?
|Statement statement = null;
|ResultSet resultSet = null;
|
|// connection is managed by a connection pool
|try {
|if(connection == null) {
|getConnection();
|}
If 'connection' is a local, what does 'getConnection' do?
If 'connection' is /not/ a local, then you are asking for threading trouble.
|if(connection != null) {
|statement = connection.createStatement();
|resultSet = statement.executeQuery(SELECT * FROM +
| table + ORDER BY idNumber);
|if(resultSet != null) {
|/* process resultSet */
|}
|resultSet.close();
|}
|statement.close();
|}
|return /* results Vector */
|}
|catch (SQLException sqle) {
|/* debug messages */
|}
|catch (Exception e) {
|/* debug messages */
|}
|finally {
|/* debug messages */
|connection = null;
You need to have statement.close() and resultSet.close() in here (in the
finally block), not up above. Also, most database connection pools
require that you call connection.close() to return the connection to the
pool. Are you ever calling connection.close()?
Also, which connection pool do you use?
|}
|return null;
|}
|
| NOTE: by checking debug logs, I can tell that during normal use there
| are NO exceptions occurring, and the finally is NOT getting called - yet
| the memory leak is still going on.
The finally block had /better/ be called: it's a requirement of the JVM
and the language.
- -chris
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEARECAAYFAkic0ccACgkQ9CaO5/Lv0PCO9gCgvHVnhOhPwIXl0qp1NOYW5Iv8
QigAn01hq4VakqxDOKV239NJujm9t/8b
=9gnQ
-END PGP SIGNATURE-
-
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Would like to monitor memory use offline
Hi, the manager application can return values in XML. You just need to hit:- http://localhost:8080/manager/status?XML=true or wherever it's located. This makes it a bit easier to write shell scripts based on Free Heap or Max Threads or whatever. I've got a perl script which checks whatever values you need and logs them to a .csv for easy reporting. I'll make it a bit better and post it if anyone has any interest. Kind regards, Phil. 2008/8/6 Richard S. Huntrods [EMAIL PROTECTED]: To Ben Mark, Thanks for the replies. I've been trying various things based on your suggestions. I find my best tool at the moment is Lambda Probe - though it hasn't been updated in a while, it does show me all the things (for the most part) that I wanted to watch. There's definitely some kind of problem with the JVM not releasing the PS Old Gen memory. I can watch as this gets gradually used up until it hits the max value - then Tomcat crashes with the GC/Heap error message. If I stop and start Tomcat before this limit is reached, the memory use resets to 0 and the system does not crash. What is puzzling is that this identical system was working for weeks on end without a problem under the older versions of Java, Tomcat and Mysql I posted in my original post - and with 1/2 the stack. Only when I moved to the new servers (with all the new versions) did the GC crashes start. Now I'm trying different GC managers to see if one of them works better at freeing up memory - like maybe the compation parallel GC. We'll see. Again, thanks for the relies. Cheers, -Richard - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Would like to monitor memory use offline
Well, thanks to Lamda probe and then to jmap, I have found my memory leak. Here are the gory details in case anyone is interested. It's not really a Tomcat issue, but rather the boundary between Tomcat and MySQL: I have a memory leak in my application, and jmap shows me that all my objects of type 'com.mysql.jdbc.Field' are not being released. In my code I was calling resultSet.close(), but not statement.close(). Doing my homework, I found the forum postings about this and so corrected my code to include statement.close() as well as resultSet.close() after my database operations. The problem is, even though I verified (debug statements) that the call is being made, the memory is STILL not being released. If I run the same large query 3 times in a row, my memory use triples. Even if I log off the session (my application) and invalidate the session, the memory is still locked up and cannot be freed by the JVM. My application is java servlets, using Java 1.6.0_06-b02, Tomcat 6.0.15 with mysql-connector-java-5.1.6-bin.jar and Mysql 5.0.51b. So what am I missing? I was sure that adding the code to close my statements would fix the problem. UPDATE: OK. In a way, I've fixed the problem. I checked my source code, and I'm now issuing the correct close() statements as required, so the memory leak is a puzzle. BUT - I went back through past versions of the jdbc connector because *** this application used to work without any problems ***. Here's what I found. (GOOD means no memory leaks at all - even under extended testing; BAD means memory leaks immediately). GOOD: mysql.jar (221 kb, 10/7/2003) mysql-connector-java-3.0.16-ga-bin.jar (231 kb, 11/16/2004) mysql-connector-java-3.0.17-ga-bin.jar (241 kb, 6/22/2005) BAD (memory leak): mysql-connector-java-3.1.10-bin.jar (409 kb, 6/23/2005) mysql-connector-java-3.1.14-bin.jar (449 kb, 10/18/2006) ... (other versions) mysql-connector-java-5.1.6-bin.jar (687 KB, 3/5/2008) So - does anyone know what the major change was between 3.0.17 and 3.1.10 that would have such a dramatic effect? Again, I realize this is not strictly a Tomcat problem (tomcat version is pretty much irrelevant to the problem), but many using Tomcat also use MySQL, so it's still of interest. Cheers, -Richard - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Would like to monitor memory use offline
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Richard,
Richard S. Huntrods wrote:
| In my code I was calling resultSet.close(), but not statement.close().
That'll do it.
| The problem is, even though I verified (debug statements) that the call
| is being made, the memory is STILL not being released. If I run the same
| large query 3 times in a row, my memory use triples. Even if I log off
| the session (my application) and invalidate the session, the memory is
| still locked up and cannot be freed by the JVM.
Can you post some (sanitized) code? Maybe you're still missing something.
Remember that you can't just close the statement when you're done with
your method... if you use multiple Statements and ResultSets, you have
to close them as you so. You also have to make sure that you have a
finally block that closes them in case of an exception condition.
I've posted it before, but I'll to it again because it bears repeating:
here is the proper way to write JDBC code:
Connection conn = null;
~PreparedStatement ps = null;
~ResultSet rs = null;
~try
~{
~conn = getConnection(); // however you get your connections
~ps = conn.prepareStatement(...); // your query
// set params, etc.
~rs = ps.executeQuery();
// do whatever you need to do with your ResultSet
~}
~finally
~{
if(null != rs) { rs.close(); } catch (SQLException sqle)
{ /* log this error! */ }
if(null != ps) { ps.close(); } catch (SQLException sqle)
{ /* log this error! */ }
if(null != conn) { conn.close(); } catch (SQLException sqle)
{ /* log this error! */ }
~}
I wrote a method in a utility class that does everything in the finally
block -- it makes the code much easier to read IMO.
Note that if you need multiple Statements, you should either use them
serially, and close each statement before you move onto the next one
(and re-use the local variable), or you should define them all outside
of the try/finally block and make sure they are closed under all
conditions. If you don't do this, you'll leak memory.
If you are using transactions, remember that you must catch /every
exception that can possibly occur/ (yes, even Errors and
RuntimeExceptions) and make sure you do a rollback before you close
everything.
| So what am I missing? I was sure that adding the code to close my
| statements would fix the problem.
Is it possible that the code you're looking at is good, but some other
code has a similar leak but is yet to be fixed.
| So - does anyone know what the major change was between 3.0.17 and
| 3.1.10 that would have such a dramatic effect?
http://dev.mysql.com/doc/refman/5.0/en/cj-news.html
It's possible that newer versions of the JDBC driver more properly
adhere to the JDBC specification which results in memory leakage
(because JDBC expects you to code in very rigid ways, including cleaning
up your own memory messes ;).
I use mysql-connector-5.0.8.jar and have not had a problem in a very
long time.
- -chris
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEARECAAYFAkiblQgACgkQ9CaO5/Lv0PBDhQCfX5Y2RkgxPChd2kGXlxQssMWE
CWkAn2JMpZbdm6knUqZSiWlE3grEepe6
=Qis0
-END PGP SIGNATURE-
-
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Would like to monitor memory use offline
To Ben Mark, Thanks for the replies. I've been trying various things based on your suggestions. I find my best tool at the moment is Lambda Probe - though it hasn't been updated in a while, it does show me all the things (for the most part) that I wanted to watch. There's definitely some kind of problem with the JVM not releasing the PS Old Gen memory. I can watch as this gets gradually used up until it hits the max value - then Tomcat crashes with the GC/Heap error message. If I stop and start Tomcat before this limit is reached, the memory use resets to 0 and the system does not crash. What is puzzling is that this identical system was working for weeks on end without a problem under the older versions of Java, Tomcat and Mysql I posted in my original post - and with 1/2 the stack. Only when I moved to the new servers (with all the new versions) did the GC crashes start. Now I'm trying different GC managers to see if one of them works better at freeing up memory - like maybe the compation parallel GC. We'll see. Again, thanks for the relies. Cheers, -Richard - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Would like to monitor memory use offline
Richard S. Huntrods wrote: SO - my question - is there a relatively easy way to create something (say a servlet) to watch the stack *just like I can do manually using the manager application* but email me when the stack approaches the memory limits? Richard, Tomcat is open source so if you want to copy something Tomcat is doing all you need to do is look at the Tomcat source and copy it. Granted it can be a bit impenetrable in places but I don't think this is one of them. 1. You know from the manager page what URL you are interested in. 2. The web.xml for the manager app will tell you which servlet is serving that URL. 3. The Tomcat 6 source is in a single tree under http://svn.apache.org/repos/asf/tomcat/tc6.0.x/trunk/java/ so finding the class is easy. 4. A quick look at the doGet() method in that class points you towards StatusTransformer.writeVMState() 5. And there you have your answer - a simple API call. http://java.sun.com/javase/6/docs/api/java/lang/Runtime.html#freeMemory() For the e-mail side, take a look at the JavaMail example in the JNDI docs. http://tomcat.apache.org/tomcat-6.0-doc/jndi-resources-howto.html Mark - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Would like to monitor memory use offline
On Sat, 2008-08-02 at 19:18 -0700, Richard S. Huntrods wrote: SO - my question - is there a relatively easy way to create something (say a servlet) to watch the stack *just like I can do manually using the manager application* but email me when the stack approaches the memory limits? Hi Richard, I'd suggest running the tomcat JVM with verbose garbage collection enabled eg. [1], then monitoring the log produced with a shell script. This should show you when you are nearing your out of memory limit. To track down the memory leak, try something like Lambda Probe [2] to give you an overview of resource consumption and monitor it over time. Cheers, Ben [1] -XX:+PrintGCTimeStamps -Xloggc:/var/log/java/gc.log -verbosegc -XX:+PrintGCDetails [2] http://www.lambdaprobe.org/d/index.htm - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Would like to monitor memory use offline
You can write a simple JSP which will run a freeMemory/totalMemory call in your JVM and possibly send a mail/log when the limits are reached. You could set a refresh interval and have this page refresh say every 5 minutes in your browser. Alternatively you can tweak with the manager app code. If you are an administrator ten from a long term monitoring aspect you may want to explore the usage of Lambdaprobe. http://www.lambdaprobe.org -Sameer --- On Sun, 8/3/08, Richard S. Huntrods [EMAIL PROTECTED] wrote: From: Richard S. Huntrods [EMAIL PROTECTED] Subject: Would like to monitor memory use offline To: users@tomcat.apache.org Date: Sunday, August 3, 2008, 7:48 AM I've been running Tomcat for many versions now, mostly without incident. However with the latest set of upgrades rather forced upon me all at once (instead of managed more properly), my application appears to have a severe memory leak. System Info: OS is Solaris 10-u5 (2008); java 1.6.0_06-b02; apache-tomcat-6.0.16; mysql 5.0.51a-solaris10-x86_64. I have fast servers and plenty of memory (8 gigs). I'm running 1 gig stack and getting at least 2 GC/stack exceptions per day (sometimes more). Yes - it's a user/use triggered leak but I can't trace it further yet. Of course what is odd is that there was NO memory leak using older versions of this stuff (Solaris 10 (2006), java 1.5.x, tomcat 5.5.12, mysql 5.0.16). I'm sure the memory leak was there, but it was well masked. On the older system I was running 512 meg stack and it never gave GC or stack errors. So, while I am actively trying to fix the memory leak, I still have to maintain these production servers at operatonal status (politics - don't ask). However, it's difficult as the memory leak is causing repeated GC and out of stack exceptions. What I've noticed recently is that when using the manager application, I can watch the memory utilization grow and more memory get allocated (via refreshing the page), right up until the stack is used up and the main application crashes. However, if I'm watching it grow, and then log on to the server and reset tomcat (stop and then start tomcat), the memory use is back at the start. Thanks to session persistence, no users are harmed during this exercise. So for the moment, while I try and debug the application, I can keep things running by having a cron job periodically reset tomcat for me. But this is really crude. Until I fix the memory leak, I'd like something a little bit more elegant. SO - my question - is there a relatively easy way to create something (say a servlet) to watch the stack *just like I can do manually using the manager application* but email me when the stack approaches the memory limits? Thanks, -Richard - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
