I have a webapp running under Jetty 8 using Derby 10.8.1.2. It runs fine
for a few weeks and then grinds to a halt (which on a Windows box also
grinds Windows to a halt). It sometimes crashes with an OutOfMemoryError
with a message that it can't create any more native threads (and I don't
get a heap dump when this happens, despite requesting one with the
HeapDumpOnOutOfMemoryError).
I've managed to get use JMX to get a heap dump and analyse it with the
Eclipse Memory Analyser. This is before it crashes, after about a week
of operation, and shows about 7MB tied up in a ConcurrentCache object.
I'm not familiar with the internals of Derby implementation, but the
leak suspect report from Eclipse is below.
Is there anyone who can help me understand what is going on here?
Eclipse Memory Analyser - Leak Suspects Report
----------------------------------------------
Problem Suspect 1 Description
One instance of "org.apache.derby.impl.services.cache.ConcurrentCache"
loaded by "org.eclipse.jetty.webapp.WebAppClassLoader @ 0x84ebf710"
occupies 7,436,248 (45.27%) bytes.
The memory is accumulated in one instance of
"org.apache.derby.impl.services.cache.ConcurrentCache"
loaded by "org.eclipse.jetty.webapp.WebAppClassLoader @ 0x84ebf710".
Shortest Paths To the Accumulation Point
Class Name Shallow Heap Retained Heap
org.apache.derby.impl.services.cache.ConcurrentCache @ 0x84fe4140
40 7,436,248
pageCache org.apache.derby.impl.store.raw.data.StoredPage @ 0x854b6978
192 5,424
page org.apache.derby.impl.store.access.btree.BranchControlRow @
0x854bde80 72 272
<Java Local> java.lang.Thread @ 0x85cf7c38 qtp22069910-7932 Thread
104 179,504
pageCache org.apache.derby.impl.store.raw.data.StoredPage @ 0x85999ab8
192 8,616
pageCache org.apache.derby.impl.store.raw.data.BaseDataFileFactoryJ4 @
0x84ff1218 152 888
cacheManager org.apache.derby.impl.services.cache.BackgroundCleaner @
0x84ff15f8 32 592
pageCache org.apache.derby.impl.store.raw.data.StoredPage @ 0x85d23440
192 34,192
pageCache org.apache.derby.impl.store.raw.data.StoredPage @ 0x8551ebe8
192 6,200
pageCache org.apache.derby.impl.store.raw.data.StoredPage @ 0x8551ed88
192 34,848
Total: 7 entries
Accumulated Objects
Class Name Shallow Heap Retained Heap Percentage
org.apache.derby.impl.services.cache.ConcurrentCache @ 0x84fe4140 40
7,436,248 45.27%
org.apache.derby.impl.services.cache.CacheEntry @ 0x851b88d8 32
49,320 0.30%
org.apache.derby.impl.services.cache.CacheEntry @ 0x85254a98 32
49,200 0.30%
org.apache.derby.impl.services.cache.CacheEntry @ 0x851b88b8 32
49,080 0.30%
org.apache.derby.impl.services.cache.CacheEntry @ 0x859c2340 32
48,928 0.30%
org.apache.derby.impl.services.cache.CacheEntry @ 0x85ac13b0 32
46,632 0.28%
org.apache.derby.impl.services.cache.CacheEntry @ 0x85093300 32
45,544 0.28%
org.apache.derby.impl.services.cache.CacheEntry @ 0x850e3068 32
44,400 0.27%
org.apache.derby.impl.services.cache.CacheEntry @ 0x850e30f8 32
40,440 0.25%
org.apache.derby.impl.services.cache.CacheEntry @ 0x850e2ef8 32
39,120 0.24%
org.apache.derby.impl.services.cache.CacheEntry @ 0x851b8e50 32
39,064 0.24%
org.apache.derby.impl.services.cache.CacheEntry @ 0x850e2e50 32
38,464 0.23%
org.apache.derby.impl.services.cache.CacheEntry @ 0x850e2d18 32
38,400 0.23%
org.apache.derby.impl.services.cache.CacheEntry @ 0x851b88f8 32
38,104 0.23%
org.apache.derby.impl.services.cache.CacheEntry @ 0x850e3150 32
37,080 0.23%
org.apache.derby.impl.services.cache.CacheEntry @ 0x851b89c8 32
36,480 0.22%
org.apache.derby.impl.services.cache.CacheEntry @ 0x859d0600 32
36,000 0.22%
org.apache.derby.impl.services.cache.CacheEntry @ 0x85a10f28 32
35,400 0.22%
org.apache.derby.impl.services.cache.CacheEntry @ 0x85d2c0e8 32
35,184 0.21%
org.apache.derby.impl.services.cache.CacheEntry @ 0x850e2dc0 32
35,104 0.21%
org.apache.derby.impl.services.cache.CacheEntry @ 0x8551d248 32
35,072 0.21%
Total: 20 entries 640
817,016 0.05
Accumulated Objects by Class
Label Number Of Objects Used Heap Size Retained
Heap Size
org.apache.derby.impl.services.cache.CacheEntry 717
22,944 7,371,592
java.util.concurrent.ConcurrentHashMap 1
40 23,872
org.apache.derby.impl.services.cache.ClockPolicy$Holder 717
17,208 17,208
org.apache.derby.iapi.store.raw.PageKey 702
16,848 16,872
org.apache.derby.iapi.store.raw.ContainerKey 146
3,504 3,504
org.apache.derby.impl.services.cache.ClockPolicy 1
32 3,104
java.lang.String 1
24 56
Total: 7 entries 2,285
60,600 7,436,208
TIA,
------------------------------------------------------------------------
John English | My old University of Brighton home page is still here:
| http://www.cem.brighton.ac.uk/staff/je/
------------------------------------------------------------------------
