[ http://issues.apache.org/jira/browse/DERBY-210?page=all ]
Deepa Remesh updated DERBY-210:
-------------------------------
Attachment: derby-210-patch5-v1.diff
derby-210-patch5-v1.status
Attaching a patch 'derby-210-patch5-v1.diff' which removes the actual memory
leaks. The previous patches were doing preliminary work to fix the client-side
finalization and network server object re-use. This patch removes the actual
memory leaks.
-----------------------------------------------------------------
Summary of patch:
-----------------------------------------------------------------
1. Eliminates the below references to PreparedStatement objects by using
WeakHashMap instead of LinkedList. When there are no other references to the
keys in a WeakHashMap, they will get removed from the map and can thus get
garbage-collected. They do not have to wait till the Connection object is
collected.
- 'openStatements_' in org.apache.derby.client.am.Connection
- 'CommitAndRollbackListeners_' in org.apache.derby.client.am.Connection
2. Updates the following comment for openStatements_:
// Since DERBY prepared statements must be re-prepared after a commit,
// then we must traverse this list after a commit and notify statements
// that they are now in an un-prepared state.
final java.util.LinkedList openStatements_ = new java.util.LinkedList();
In the code, I did not see this list being traversed after a commit to
re-prepare statements. Also, I think this is not needed since Derby does not
require re-prepare of statements after a commit. Currently, this list is used
to close all open statements when the originating connection is closed.
3. Removes all ResultSets from HashTable
'positionedUpdateCursorNameToResultSet_' in SectionManager. Only result sets of
positioned update statements were being removed from this hashtable whereas all
result sets were added. Because of this, client driver was holding on to result
sets and statements even after rs.close() was called.
4. Modifies the test jdbcapi/derbyStress.java to run the test for derby-210.
The test was checked in as patch2 but disabled for client framework.
-----------------------------------------------------------------
Open items related to this issue
-----------------------------------------------------------------
* http://issues.apache.org/jira/browse/DERBY-817 (Improvements to Network
Client driver - analyze/improve use of java collection classes in the code) -
This issue lists some potential problem areas and cleanup tasks identified in
the client driver code.
* http://issues.apache.org/jira/browse/DERBY-1002 (Check that DRDAStatement and
DRDAResultSet states are reset when they are re-used) - This issue was blocking
DERBY-210. Patch for the blocking issue has been committed. There is still some
minor cleanup to be done in this issue.
* http://issues.apache.org/jira/browse/DERBY-1021 (Perform cleanup actions
which require synchronization outside the finalizer) - This issue is linked to
("is part of") DERBY-210. Maybe, it is easier if I had made this a sub-task of
DERBY-210. I will close DERBY-1021 and open a sub-task for 210 with details
about work needed on the finalizer.
With this patch, I ran the following tests on Windows XP:
* derbynetclientmats suite 6 times on my machine (3 times with Sun JDK1.4.2 and
3 times with Sun JDK1.5). No failures.
* repro with 100,000 statements and checked the memory usage on client and
server. No spikes in memory usage. Without this patch, the repro used to give
OutOfMemory error after executing 950 statements. Tests run with Sun JDK1.4.2
with 64M heap space .
* test to compare memory usage when statements_are_not_explicitly_closed vs
statements_are_explicitly_closed. Tests run with Sun JDK1.4.2 with 64M heap
space.
case1: repro with 5000 statements
case2: modified_repro_which_explicitly_closes_prepared_statements with 5000
statements.
I compared the memory usage on client and server in these two cases and saw
that memory usage over time is similar. Memory usage is calculated as
Runtime.totalMemory() - Runtime.freeMemory().
I would appreciate if someone can take a look at this patch.
> Network Server will leak prepared statements if not explicitly closed by the
> user until the connection is closed
> ----------------------------------------------------------------------------------------------------------------
>
> Key: DERBY-210
> URL: http://issues.apache.org/jira/browse/DERBY-210
> Project: Derby
> Type: Bug
> Components: Network Client
> Reporter: Kathey Marsden
> Assignee: Deepa Remesh
> Attachments: DOTS_ATCJ2_Derby-noPatch.png, DOTS_ATCJ2_Derby-withPatch.png,
> derby-210-patch1.diff, derby-210-patch2.diff, derby-210-patch2.status,
> derby-210-patch3.diff, derby-210-patch4-v2.diff, derby-210-patch4-v2.status,
> derby-210-patch4-v3.diff, derby-210-patch4-v3.status,
> derby-210-patch5-v1.diff, derby-210-patch5-v1.status,
> derby-210-v2-draft.diff, derby-210-v2-draft.status, derbyStress.java
>
> Network server will not garbage collect prepared statements that are not
> explicitly closed by the user. So a loop like this will leak.
> ...
> PreparedStatement ps;
> for (int i = 0 ; i < numPs; i++)
> {
> ps = conn.prepareStatement(selTabSql);
> rs =ps.executeQuery();
> while (rs.next())
> {
> rs.getString(1);
> }
> rs.close();
> // I'm a sloppy java programmer
> //ps.close();
> }
>
> To reproduce run the attached program
> java derbyStress
> Both client and server will grow until the connection is closed.
>
> It is likely that the fix for this will have to be in the client. The client
> does not send protocol to close the prepared statement, but rather reuses the
> PKGNAMCSN on the PRPSQLSTT request once the prepared statement has been
> closed. This is how the server knows to close the old statement and create a
> new one.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
