[jira] Commented: (DERBY-3596) Creation of logical connections from a pooled connection causes resource leak on the server
[ https://issues.apache.org/jira/browse/DERBY-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12603821#action_12603821 ] Knut Anders Hatlen commented on DERBY-3596: --- 5a looks good to me. I agree that it's better to keep this code simple and that 5a is preferable to 4a. > Creation of logical connections from a pooled connection causes resource leak > on the server > --- > > Key: DERBY-3596 > URL: https://issues.apache.org/jira/browse/DERBY-3596 > Project: Derby > Issue Type: Bug > Components: Network Client, Network Server, Performance >Affects Versions: 10.1.3.1, 10.2.2.0, 10.3.2.1, 10.4.1.3, 10.5.0.0 >Reporter: Kristian Waagan >Assignee: Kristian Waagan > Attachments: complex-fix-heap.png, ConnectionPoolingBug.java, > derby-3596-1a-complex_approach.diff, derby-3596-1b-complex_approach.diff, > derby-3596-2a-simple_approach.diff, derby-3596-3a-test_cleanup.diff, > derby-3596-4a-complex_check_creds.diff, > derby-3596-5a-complex_skip_creds.diff, nofix-heap.png, simple-fix-heap.png > > > When using ClientConnectionPoolDataSource and connection pooling, a new > connection / transaction is created for every new logical connection, and the > resources are not freed / cleaned up in the server. They are not even cleaned > up when the physical connection (ClientPooledConnection) is closed. > A logical connection is obtained by invoking > ClientPooledConnection.getConnection(). > I have observed that if you run the repro enough times against the same > server, the number of transaction in the transaction table will be reduced > now and then. I believe this is garbage collection, but I have not > investigated the problem enough to tell for sure what's going on. > I have also seen some locks not being freed, causing timeouts in some > applications. I don't have a repro for the lock problem at this time, but it > is very likely related to this issue. > Note that XA connections are handled differently on the server, and do > probably not have this problem. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-3596) Creation of logical connections from a pooled connection causes resource leak on the server
[ https://issues.apache.org/jira/browse/DERBY-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12602368#action_12602368 ] Kristian Waagan commented on DERBY-3596: Regression tests completed without failures with patch 5a applied (Sun JDK 1.6, Solaris 10). > Creation of logical connections from a pooled connection causes resource leak > on the server > --- > > Key: DERBY-3596 > URL: https://issues.apache.org/jira/browse/DERBY-3596 > Project: Derby > Issue Type: Bug > Components: Network Client, Network Server, Performance >Affects Versions: 10.1.3.1, 10.2.2.0, 10.3.2.1, 10.4.1.3, 10.5.0.0 >Reporter: Kristian Waagan >Assignee: Kristian Waagan > Attachments: complex-fix-heap.png, ConnectionPoolingBug.java, > derby-3596-1a-complex_approach.diff, derby-3596-1b-complex_approach.diff, > derby-3596-2a-simple_approach.diff, derby-3596-3a-test_cleanup.diff, > derby-3596-4a-complex_check_creds.diff, > derby-3596-5a-complex_skip_creds.diff, nofix-heap.png, simple-fix-heap.png > > > When using ClientConnectionPoolDataSource and connection pooling, a new > connection / transaction is created for every new logical connection, and the > resources are not freed / cleaned up in the server. They are not even cleaned > up when the physical connection (ClientPooledConnection) is closed. > A logical connection is obtained by invoking > ClientPooledConnection.getConnection(). > I have observed that if you run the repro enough times against the same > server, the number of transaction in the transaction table will be reduced > now and then. I believe this is garbage collection, but I have not > investigated the problem enough to tell for sure what's going on. > I have also seen some locks not being freed, causing timeouts in some > applications. I don't have a repro for the lock problem at this time, but it > is very likely related to this issue. > Note that XA connections are handled differently on the server, and do > probably not have this problem. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-3596) Creation of logical connections from a pooled connection causes resource leak on the server
[
https://issues.apache.org/jira/browse/DERBY-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12600499#action_12600499
]
Kristian Waagan commented on DERBY-3596:
- 1) I moved the reset to before "if (appRequester != null)". It should be true
for the shortest time possible, but it must survive for a few iterations in the
processCommands loop after it has been set to true.
- 2) Sorry, I was at bit unclear. If you look at the end of the method
processCommands, you see this inside a SanityManager.DEBUG block:
if (!this.deferredReset && pbsd != null) {
...
SanityManager.ASSERT(!pbsd.isModified(),
This assert fails because of the connection reset I do. When the PBSD is
written, we're on track again and the flag can be reset and we execute the
assert again for later iterations in the processCommands loop. This workaround
is required due to the "hackyness" of the fix...
- 3) No, I don't believe it is entirely safe to skip the user/password check.
However, that code is what creates the new connections, which is what we want
to avoid. The credentials are verified by actually connecting to the database.
Some options:
a) Make an extra connection attempt to verify credentials.
b) Hang on to the credentials made during the initial connection and see if
they have changed for the deferred resets.
Note that our driver had the functionality to change the credentials, but
it doesn't anymore. So I suppose the ability to exploit this depends on if a
malicious client can "take over the TCP-IP pipe" and act as the initial user.
The new user credential won't take effect.
For the general case, this will cause some overhead, and it is my opinion
that the driver shouldn't send this "reconnection sequence" at all. I'll try to
implement option b) in the next patch, along with some more comments and
references to the Jira issue.
> Creation of logical connections from a pooled connection causes resource leak
> on the server
> ---
>
> Key: DERBY-3596
> URL: https://issues.apache.org/jira/browse/DERBY-3596
> Project: Derby
> Issue Type: Bug
> Components: Network Client, Network Server, Performance
>Affects Versions: 10.1.3.1, 10.2.2.0, 10.3.2.1, 10.4.1.3, 10.5.0.0
>Reporter: Kristian Waagan
>Assignee: Kristian Waagan
> Attachments: complex-fix-heap.png, ConnectionPoolingBug.java,
> derby-3596-1a-complex_approach.diff, derby-3596-1b-complex_approach.diff,
> derby-3596-2a-simple_approach.diff, derby-3596-3a-test_cleanup.diff,
> nofix-heap.png, simple-fix-heap.png
>
>
> When using ClientConnectionPoolDataSource and connection pooling, a new
> connection / transaction is created for every new logical connection, and the
> resources are not freed / cleaned up in the server. They are not even cleaned
> up when the physical connection (ClientPooledConnection) is closed.
> A logical connection is obtained by invoking
> ClientPooledConnection.getConnection().
> I have observed that if you run the repro enough times against the same
> server, the number of transaction in the transaction table will be reduced
> now and then. I believe this is garbage collection, but I have not
> investigated the problem enough to tell for sure what's going on.
> I have also seen some locks not being freed, causing timeouts in some
> applications. I don't have a repro for the lock problem at this time, but it
> is very likely related to this issue.
> Note that XA connections are handled differently on the server, and do
> probably not have this problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-3596) Creation of logical connections from a pooled connection causes resource leak on the server
[ https://issues.apache.org/jira/browse/DERBY-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12600462#action_12600462 ] Knut Anders Hatlen commented on DERBY-3596: --- - I don't know if this can ever happen, but since we always set deferredReset to false in parseEXCSAT() if appRequester is null, I assume that it is possible that deferredReset is true when parseEXCSAT() is called. As the code is now, it won't set deferredReset to false if it's true when the method is called and it is an XA request. Is this intended, or should deferredReset always be set to the value of (appRequester != null && !appRequester.isXARequester())? - I'm not sure I understand this comment: +// Reset the flag again. In sane builds it is used to avoid asserts, but +// we want to reset it as soon as possible to avoid masking real bugs. +this.deferredReset = false; I don't see any asserts that check deferredReset, and I don't see how it masks bugs. - Is it safe to skip the user/password check at the end of parseSECCHK() when processing a deferred reset? Earlier in that method, the user id and password fields of the database object are updated with whichever the SECCHK message contains, so even if the Derby network client never changes the user id, could a malicious client exploit this somehow? > Creation of logical connections from a pooled connection causes resource leak > on the server > --- > > Key: DERBY-3596 > URL: https://issues.apache.org/jira/browse/DERBY-3596 > Project: Derby > Issue Type: Bug > Components: Network Client, Network Server, Performance >Affects Versions: 10.1.3.1, 10.2.2.0, 10.3.2.1, 10.4.1.3, 10.5.0.0 >Reporter: Kristian Waagan >Assignee: Kristian Waagan > Attachments: complex-fix-heap.png, ConnectionPoolingBug.java, > derby-3596-1a-complex_approach.diff, derby-3596-1b-complex_approach.diff, > derby-3596-2a-simple_approach.diff, derby-3596-3a-test_cleanup.diff, > nofix-heap.png, simple-fix-heap.png > > > When using ClientConnectionPoolDataSource and connection pooling, a new > connection / transaction is created for every new logical connection, and the > resources are not freed / cleaned up in the server. They are not even cleaned > up when the physical connection (ClientPooledConnection) is closed. > A logical connection is obtained by invoking > ClientPooledConnection.getConnection(). > I have observed that if you run the repro enough times against the same > server, the number of transaction in the transaction table will be reduced > now and then. I believe this is garbage collection, but I have not > investigated the problem enough to tell for sure what's going on. > I have also seen some locks not being freed, causing timeouts in some > applications. I don't have a repro for the lock problem at this time, but it > is very likely related to this issue. > Note that XA connections are handled differently on the server, and do > probably not have this problem. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-3596) Creation of logical connections from a pooled connection causes resource leak on the server
[ https://issues.apache.org/jira/browse/DERBY-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12599334#action_12599334 ] Knut Anders Hatlen commented on DERBY-3596: --- I logged the problem with the embedded driver as DERBY-3690 so that we can address it separately. > Creation of logical connections from a pooled connection causes resource leak > on the server > --- > > Key: DERBY-3596 > URL: https://issues.apache.org/jira/browse/DERBY-3596 > Project: Derby > Issue Type: Bug > Components: Network Client, Network Server, Performance >Affects Versions: 10.1.3.1, 10.2.2.0, 10.3.2.1, 10.4.1.3, 10.5.0.0 >Reporter: Kristian Waagan >Assignee: Kristian Waagan > Attachments: complex-fix-heap.png, ConnectionPoolingBug.java, > derby-3596-1a-complex_approach.diff, derby-3596-2a-simple_approach.diff, > nofix-heap.png, simple-fix-heap.png > > > When using ClientConnectionPoolDataSource and connection pooling, a new > connection / transaction is created for every new logical connection, and the > resources are not freed / cleaned up in the server. They are not even cleaned > up when the physical connection (ClientPooledConnection) is closed. > A logical connection is obtained by invoking > ClientPooledConnection.getConnection(). > I have observed that if you run the repro enough times against the same > server, the number of transaction in the transaction table will be reduced > now and then. I believe this is garbage collection, but I have not > investigated the problem enough to tell for sure what's going on. > I have also seen some locks not being freed, causing timeouts in some > applications. I don't have a repro for the lock problem at this time, but it > is very likely related to this issue. > Note that XA connections are handled differently on the server, and do > probably not have this problem. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-3596) Creation of logical connections from a pooled connection causes resource leak on the server
[ https://issues.apache.org/jira/browse/DERBY-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12599323#action_12599323 ] Knut Anders Hatlen commented on DERBY-3596: --- Regardless of which fix we choose, I think the changes to EmbedConnection.resetFromPool() should be part of it. I tried to set the schema on a logical connection with the embedded driver, and when I closed that connection and opened a new logical connection, the current schema had not been reset. (We may consider pushing the resetting all the way into LCC.resetFromPool(), by the way.) > Creation of logical connections from a pooled connection causes resource leak > on the server > --- > > Key: DERBY-3596 > URL: https://issues.apache.org/jira/browse/DERBY-3596 > Project: Derby > Issue Type: Bug > Components: Network Client, Network Server, Performance >Affects Versions: 10.1.3.1, 10.2.2.0, 10.3.2.1, 10.4.1.3, 10.5.0.0 >Reporter: Kristian Waagan >Assignee: Kristian Waagan > Attachments: complex-fix-heap.png, ConnectionPoolingBug.java, > derby-3596-1a-complex_approach.diff, derby-3596-2a-simple_approach.diff, > nofix-heap.png, simple-fix-heap.png > > > When using ClientConnectionPoolDataSource and connection pooling, a new > connection / transaction is created for every new logical connection, and the > resources are not freed / cleaned up in the server. They are not even cleaned > up when the physical connection (ClientPooledConnection) is closed. > A logical connection is obtained by invoking > ClientPooledConnection.getConnection(). > I have observed that if you run the repro enough times against the same > server, the number of transaction in the transaction table will be reduced > now and then. I believe this is garbage collection, but I have not > investigated the problem enough to tell for sure what's going on. > I have also seen some locks not being freed, causing timeouts in some > applications. I don't have a repro for the lock problem at this time, but it > is very likely related to this issue. > Note that XA connections are handled differently on the server, and do > probably not have this problem. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-3596) Creation of logical connections from a pooled connection causes resource leak on the server
[ https://issues.apache.org/jira/browse/DERBY-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12599290#action_12599290 ] Kristian Waagan commented on DERBY-3596: Thanks for looking at the issue Kathey. I'll include the Jira number in the relevant comments when I post an updated patch. I agree that the simple fix seems less risky. I get one error when I run the regression tests with it. The failure is in one of the new tests I wrote for JDBC statement pooling, so I have to look into the test and also check how statement pooling is affected by the fix. I do have some reservations about the more complex fix. Even though the approach is more optimal, the implementation is a bit hacky due to compatibility constraints. I wonder if it would be better to design and implement a proper mechanism at a later time. Will we ever roll a compatibility breaking release? If not, we have to code this using version checking and keep multiple implementations on both the client and server side. Also, please note that there are more alternatives than the two I have suggested here. For instance, today we have Database and XADatabase. It might be possible to introduce a PoolingDatabase, taken we can detect when clients are indeed working with pooled connections. The difference between this one and the current complex fix, would be that the former would map client logical connections onto embedded logical connections, whereas the latter would map client logical connections directly onto a single embedded physical connection. This has some implications regarding connection state reset and object creation. I'll work a bit more on the test failure and getting a better grip on things. Feel free to add more comments. > Creation of logical connections from a pooled connection causes resource leak > on the server > --- > > Key: DERBY-3596 > URL: https://issues.apache.org/jira/browse/DERBY-3596 > Project: Derby > Issue Type: Bug > Components: Network Client, Network Server, Performance >Affects Versions: 10.1.3.1, 10.2.2.0, 10.3.2.1, 10.4.1.3, 10.5.0.0 >Reporter: Kristian Waagan >Assignee: Kristian Waagan > Attachments: complex-fix-heap.png, ConnectionPoolingBug.java, > derby-3596-1a-complex_approach.diff, derby-3596-2a-simple_approach.diff, > nofix-heap.png, simple-fix-heap.png > > > When using ClientConnectionPoolDataSource and connection pooling, a new > connection / transaction is created for every new logical connection, and the > resources are not freed / cleaned up in the server. They are not even cleaned > up when the physical connection (ClientPooledConnection) is closed. > A logical connection is obtained by invoking > ClientPooledConnection.getConnection(). > I have observed that if you run the repro enough times against the same > server, the number of transaction in the transaction table will be reduced > now and then. I believe this is garbage collection, but I have not > investigated the problem enough to tell for sure what's going on. > I have also seen some locks not being freed, causing timeouts in some > applications. I don't have a repro for the lock problem at this time, but it > is very likely related to this issue. > Note that XA connections are handled differently on the server, and do > probably not have this problem. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-3596) Creation of logical connections from a pooled connection causes resource leak on the server
[ https://issues.apache.org/jira/browse/DERBY-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12599175#action_12599175 ] Kathey Marsden commented on DERBY-3596: --- I am about even on the two options. The simple fix seems less risky and less likely to have bugs, but I don't know if the complex fix has other performance benefits. Just a nit on the fix. It would be good to mention the bug number in the comments where you talk about the bug, so future readers can refer to this conversation. > Creation of logical connections from a pooled connection causes resource leak > on the server > --- > > Key: DERBY-3596 > URL: https://issues.apache.org/jira/browse/DERBY-3596 > Project: Derby > Issue Type: Bug > Components: Network Client, Network Server, Performance >Affects Versions: 10.1.3.1, 10.2.2.0, 10.3.2.1, 10.4.1.3, 10.5.0.0 >Reporter: Kristian Waagan >Assignee: Kristian Waagan > Attachments: complex-fix-heap.png, ConnectionPoolingBug.java, > derby-3596-1a-complex_approach.diff, derby-3596-2a-simple_approach.diff, > nofix-heap.png, simple-fix-heap.png > > > When using ClientConnectionPoolDataSource and connection pooling, a new > connection / transaction is created for every new logical connection, and the > resources are not freed / cleaned up in the server. They are not even cleaned > up when the physical connection (ClientPooledConnection) is closed. > A logical connection is obtained by invoking > ClientPooledConnection.getConnection(). > I have observed that if you run the repro enough times against the same > server, the number of transaction in the transaction table will be reduced > now and then. I believe this is garbage collection, but I have not > investigated the problem enough to tell for sure what's going on. > I have also seen some locks not being freed, causing timeouts in some > applications. I don't have a repro for the lock problem at this time, but it > is very likely related to this issue. > Note that XA connections are handled differently on the server, and do > probably not have this problem. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-3596) Creation of logical connections from a pooled connection causes resource leak on the server
[ https://issues.apache.org/jira/browse/DERBY-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12586257#action_12586257 ] Knut Anders Hatlen commented on DERBY-3596: --- Sorry, I didn't notice that setAutoCommit(false) was commented out, so this is probably not the same issue as DERBY-3319. > Creation of logical connections from a pooled connection causes resource leak > on the server > --- > > Key: DERBY-3596 > URL: https://issues.apache.org/jira/browse/DERBY-3596 > Project: Derby > Issue Type: Bug > Components: Network Client, Network Server, Performance >Affects Versions: 10.1.3.1, 10.2.2.0, 10.3.2.1, 10.4.1.0, 10.5.0.0 >Reporter: Kristian Waagan > Attachments: ConnectionPoolingBug.java > > > When using ClientConnectionPoolDataSource and connection pooling, a new > connection / transaction is created for every new logical connection, and the > resources are not freed / cleaned up in the server. They are not even cleaned > up when the physical connection (ClientPooledConnection) is closed. > A logical connection is obtained by invoking > ClientPooledConnection.getConnection(). > I have observed that if you run the repro enough times against the same > server, the number of transaction in the transaction table will be reduced > now and then. I believe this is garbage collection, but I have not > investigated the problem enough to tell for sure what's going on. > I have also seen some locks not being freed, causing timeouts in some > applications. I don't have a repro for the lock problem at this time, but it > is very likely related to this issue. > Note that XA connections are handled differently on the server, and do > probably not have this problem. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-3596) Creation of logical connections from a pooled connection causes resource leak on the server
[ https://issues.apache.org/jira/browse/DERBY-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12585811#action_12585811 ] Knut Anders Hatlen commented on DERBY-3596: --- Isn't this the same problem as the one described in DERBY-3319? > Creation of logical connections from a pooled connection causes resource leak > on the server > --- > > Key: DERBY-3596 > URL: https://issues.apache.org/jira/browse/DERBY-3596 > Project: Derby > Issue Type: Bug > Components: Network Client, Network Server, Performance >Affects Versions: 10.1.3.1, 10.2.2.0, 10.3.2.1, 10.4.1.0, 10.5.0.0 >Reporter: Kristian Waagan > Attachments: ConnectionPoolingBug.java > > > When using ClientConnectionPoolDataSource and connection pooling, a new > connection / transaction is created for every new logical connection, and the > resources are not freed / cleaned up in the server. They are not even cleaned > up when the physical connection (ClientPooledConnection) is closed. > A logical connection is obtained by invoking > ClientPooledConnection.getConnection(). > I have observed that if you run the repro enough times against the same > server, the number of transaction in the transaction table will be reduced > now and then. I believe this is garbage collection, but I have not > investigated the problem enough to tell for sure what's going on. > I have also seen some locks not being freed, causing timeouts in some > applications. I don't have a repro for the lock problem at this time, but it > is very likely related to this issue. > Note that XA connections are handled differently on the server, and do > probably not have this problem. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-3596) Creation of logical connections from a pooled connection causes resource leak on the server
[ https://issues.apache.org/jira/browse/DERBY-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12585549#action_12585549 ] Dyre Tjeldvoll commented on DERBY-3596: --- My understanding is that the statement cache is only available when using ClientConnectionPoolDataSource. Perhaps we should include a reference to this issue in the description of the statement cache feature in the release notes? Seems like anyone wanting to try out the statement cache will likely experience this problem. > Creation of logical connections from a pooled connection causes resource leak > on the server > --- > > Key: DERBY-3596 > URL: https://issues.apache.org/jira/browse/DERBY-3596 > Project: Derby > Issue Type: Bug > Components: Network Client, Network Server, Performance >Affects Versions: 10.1.3.1, 10.2.2.0, 10.3.2.1, 10.4.1.0, 10.5.0.0 >Reporter: Kristian Waagan > Attachments: ConnectionPoolingBug.java > > > When using ClientConnectionPoolDataSource and connection pooling, a new > connection / transaction is created for every new logical connection, and the > resources are not freed / cleaned up in the server. They are not even cleaned > up when the physical connection (ClientPooledConnection) is closed. > A logical connection is obtained by invoking > ClientPooledConnection.getConnection(). > I have observed that if you run the repro enough times against the same > server, the number of transaction in the transaction table will be reduced > now and then. I believe this is garbage collection, but I have not > investigated the problem enough to tell for sure what's going on. > I have also seen some locks not being freed, causing timeouts in some > applications. I don't have a repro for the lock problem at this time, but it > is very likely related to this issue. > Note that XA connections are handled differently on the server, and do > probably not have this problem. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
