Thanks Bryan. I have tried that tool before and with the large heaps it has a
problem.
This is a network server setup and might be one of the more heavily used Derby
applications:
- database size of about 50G of network device provisioning information and
growing
- daily about 6.5M network performance records being added and the database
grows to about 200Gb before being purged. This has grown from about 1M since
last June.
- a poor man's partitioning by putting network performance records into tables
for each week of the year (53 tables)
- a View created to present these tables as one logical table that uses
is based on a RestrictedTable VTI that creates an optimizes a UNION query over
the property tables dynamically based on the query constraints
- continuous operation 7/24 including online backups done every night
- Web service interface that acts like tens to about 100 uses simultaneously
accessing the system
- Normally zero system administration is available
So far I have been able to tweak and tune and even fix some Derby bugs to keep
this running ;) We have definitely been stressing Derby!
Unfortunately out testing in our lab did not turn up these issues and it is
only in production at a customer site with zero access for me to monitor, that
these problems have shown up.
But I think I am on the right track with the earlier email to Mike about the
statement cache. And you are 100% right about getting a simpler setup if
possible to recreate the problem. Once it is possible to recreate 100% of the
time, I have no doubt a solution will be found.
Thanks for the input.
Brett
-----Original Message-----
From: Bryan Pendleton [mailto:[email protected]]
Sent: Wednesday, March 28, 2012 9:49 AM
To: [email protected]
Subject: Re: Really need some help here please. I think I am having a problem
with some sort of caching in Derby
> So what I found is that if I hit the "Perform GC" button on JvisualVM
> to perform an explicit garbage collection, the same query that was
> taking 20 seconds or more instantaneously comes down to taking about 6
> milliseconds. Its processing time then starts to creep up back towards 20
> seconds or more.
Is this an embedded configuration? Or are you looking at the Network Server?
If it's embedded, have you considered the possibility that it's something other
than Derby that's consuming the memory?
See if you can use a tool like this:
http://www.eclipse.org/mat/
to get some overall histogram of memory consumption before and after that
explicit GC.
It might also make it easier to run your experiment under a smaller JVM, with a
smaller workload, because you'll have less overall data to record and the
analysis tools will be more responsive.
But the patterns may be less clear.
thanks,
bryan
