[ 
https://issues.apache.org/jira/browse/DERBY-7001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16536505#comment-16536505
 ] 

Leo Peng commented on DERBY-7001:
---------------------------------

Hi [~bryanpendleton] and [~rhillegas],

Thank you for your reply.

My point was, I use the same code connecting to Derby and MySQL, but the 
outcomes are quite different. If you compare the two profile screenshots, you 
can see that the Derby has many unloaded classes, while MySQL does not have 
any; the Derby has frequent GC while MySQL does not; the most concerning thing 
is that the Metaspace committed size(yellow block on the top left) keeps 
growing which would consume all the Metaspace in the end(see 
OutOfMemoryError.png). 

You are right, it is not actually running queries, but it does not matter. The 
thing is it runs out of Metaspace memory. It can be a Derby bug or it also can 
be a JVM bug. With the comparison between Derby and MySQL, I guess I can say 
that there is something wrong with Derby. During the process, Derby generates 
many classes, but they are garbage collected by JVM at the same time, so the 
Metaspace should not be run out. Also, it does not make sense that the used 
Metaspace keeps steady but the committed size keeps growing until full. Thus, 
from this point of view, it tends to be a JVM issue.

> Derby JDBC driver classs loader leak
> ------------------------------------
>
>                 Key: DERBY-7001
>                 URL: https://issues.apache.org/jira/browse/DERBY-7001
>             Project: Derby
>          Issue Type: Bug
>          Components: JDBC
>    Affects Versions: 10.12.1.1
>            Reporter: Leo Peng
>            Priority: Major
>         Attachments: DerbyProfile.png, DerbyTest.zip, MySqlProfile .png, 
> OutOfMemoryError.png
>
>
> We had a customer facing Metaspace out of memory issue. After investigation. 
> I found that the Derby JDBC driver may have a class loader leak problem. The 
> reason is that I use same code doing the same thing with Derby and MySql, 
> however, the statistics showed that there is a lot of difference. What I did 
> was using Derby or MySql to create some tables and then do 200k queries on 
> those tables. and collect some statistics from them.
> When using Derby as the data source, it is very easy to run out of Metaspace. 
> Derby loads a large number of classes and committed Metaspace keeps 
> increasing until full, at the same time, the GC operations are very frequent. 
> On the contrary, when using MySql as the data source, it uses steady 
> Metaspace during the process and no extra class loaded to JVM and no GCs.
> Another thing I found was that it took Derby more than 10 minutes to run 200k 
> queries while it only costs MySql less than a second.
> Please see the attached screenshot profiles for more details.
> The JVM arguments I was using:
> {noformat}
> -XX:MaxMetaspaceSize=96m
> -XX:+HeapDumpOnOutOfMemoryError
> -XX:HeapDumpPath=C:/heapdump
> -verbose:gc
> -verbose:class
> {noformat}
> Some GCs when using Derby
> {noformat}
> [GC (Allocation Failure)  134632K->12584K(245760K), 0.0063202 secs]
> [GC (Metadata GC Threshold)  32171K->12432K(241152K), 0.0045418 secs]
> [Full GC (Metadata GC Threshold)  12432K->8691K(238592K), 0.0405280 secs]
> [GC (Allocation Failure)  128499K->12435K(237056K), 0.0074788 secs]
> [GC (Metadata GC Threshold)  38054K->12531K(234496K), 0.0053295 secs]
> [Full GC (Metadata GC Threshold)  12531K->8719K(232448K), 0.0366468 secs]
> [GC (Allocation Failure)  122895K->12271K(229376K), 0.0087422 secs]
> [GC (Metadata GC Threshold)  43475K->12799K(224768K), 0.0055041 secs]
> [Full GC (Metadata GC Threshold)  12799K->8738K(223744K), 0.0333892 secs]
> [GC (Allocation Failure)  117794K->12258K(223232K), 0.0060031 secs]
> [GC (Metadata GC Threshold)  48433K->12674K(221184K), 0.0050477 secs]
> [Full GC (Metadata GC Threshold)  12674K->8759K(219136K), 0.0339425 secs]
> [GC (Allocation Failure)  112695K->12183K(216064K), 0.0066411 secs]
> [GC (Metadata GC Threshold)  54733K->12623K(212992K), 0.0052730 secs]
> [Full GC (Metadata GC Threshold)  12623K->8778K(210432K), 0.0387252 secs]
> {noformat}
> I also attached the Java testing project for your reference.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to