I've seen this behavior with Hibrnate and DB2. The memory usage would
ramp up out of control if we didn't properly close our Hibernate
sessions. We thought we were but they changed the way sessions were
managed between the release we were using and the last time they updated
their docs. What was sad is we saw this behavior with as little as 2
users connected at a time. My guess is when he ran it with MySQL the
sessions were getting closed becasue MySQL's driver was automaticaly
closing old connections. I recall that from when I used MySQL. It's just
lazy coding if you ask me, closing a connection should be the
responsibility of the one who opened it.
I am certain that the issue is with Hibernate, or possibly
Spring/Hibernate combination. It can be difficult to track issues down
with those frameworks.
p
[EMAIL PROTECTED] wrote:
Well that's the thing.
He's using Derby in conjunction with hibernate and spring.
My guess is that its something in the combination of the tools that is
causing the issue.
That's why I was wondering if he just tried the queries against the database
independent of the app.
-----Original Message-----
From: Mike Matrigali [mailto:[EMAIL PROTECTED]
Sent: Thursday, August 31, 2006 6:55 PM
To: Derby Discussion
Subject: Re: Concurrent access problems?
Does anyone know how the resource described below as a "process handle"
or "Semaphore", relates to actual things that Derby actively uses?
Actual Derby locking is done by user object allocation so I don't think
it would show as such in this tool. It uses java synchronization
blocks, but it is up to the jvm to release resources based on this usage.
Once the page cache fills up Derby should not really accumulate much
more resources no matter what the number of rows. For instance
(again once the page cache is filled), the accumulated resources to
scan a one row table are expected to be about the same as to scan
a 10 million row table. Depending on datatypes there may be object
allocation per row, but the objects should be freed after the row
is done. For most of the datatypes derby is coded to reuse the
objects for each row so the normal case is once the scan starts no
object allocation is necessary.
Just to understand where the problem is, do observe that the resources
are still allocated after the query is finished and the transaction
has committed? How about after the connection is closed (but the db
still opened)? And finally how about after the db is closed (using
shutdown=true?).
Mark Hellkamp wrote:
<[EMAIL PROTECTED]> writes:
[mjs]
The person who created the JIRA issue is the one who said "Large amounts
of
data".
10K rows of 9 columns, is not a "large amount" of data.
While Mark posted that he tried using MySQL instead of Derby, one has to
ask
if he tried running the query independent of his app.
The point is that there are a lot of variables that can impact
performance.
Spring and Hibernate are two of them.
So unless someone introduced a "bug" in the latest release, I'm a bit
skeptical of the issue.
The application is querying about 24 tables in all for each request that
comes
in. The number of rows in the tables range from 0 to over 30,000 in my
current
testing. As I add more concurrent request threads the database continues
to
allocate additional process handles (this is on Windows). Using Process
Explorer
(from www.sysinternals.com) I can see that the Derby process is
allocating
Semaphores and does not appear to ever release them. As more data is
added to
the tables, the process handle count continues to grow even though the
results
of the queries remain the same. Derby appears to be allocating resources
(locks?) but never cleaning them up.
