Suresh Subbiah created TRAFODION-2218:
-----------------------------------------
Summary: Memory leak from JVM used for TMUDF compile time
interaction
Key: TRAFODION-2218
URL: https://issues.apache.org/jira/browse/TRAFODION-2218
Project: Apache Trafodion
Issue Type: Bug
Components: sql-cmp
Affects Versions: 2.0-incubating
Reporter: Suresh Subbiah
Assignee: Suresh Subbiah
Fix For: 2.1-incubating
JVM heap size for the JNI JVM attached to master executor grows when the
executor is used to prepare queries with a TMUDF that have compile time
interaction methods defined.
This can seen with the sessionize_java TMUDF that is part of the regression
framework. Please run regress/udr/TEST001 (after commenting out the clean_up
command at the end of the test). Then from a new sqlci session do
set schema sch;
prepare s from
select * from udf(sessionize_java(table(select * from t001_Datatypes),
'C_VARCHAR', 'C_DECIMAL_UNSIGNED', 60));
-- repeat prepare once more
-------------- DO heap analysis as shown below
-- repeat prepare 10 more times
------------- DO heap analysis as shown below
>>>>>>>
Heap analysis
-- After 2 prepares
jmap -histo:live <sqlci-pid> > histo0
-- After 12 prepares
jmap -histo:live <sqlci-pid> > histo1
Second column below is num_occurences and third column is size in bytes
[ssubbiah@edev08 hive]$ grep ReturnInfo histo*
histo0: 221: 12 384
org.trafodion.sql.udr.LmUDRObjMethodInvoke$ReturnInfo
histo1: 94: 72 2304
org.trafodion.sql.udr.LmUDRObjMethodInvoke$ReturnInfo
[ssubbiah@edev08 hive]$ grep "\[B" histo* | grep -v "\[\["
histo0: 7: 14660 1595584 [B
histo1: 6: 14714 2302296 [B
This shows that the nested static class
.LmUDRObjMethodInvoke$ReturnInfo is leaking. The leak rate is 6 instances per
statement compile. This class has byte arrays as data members which are used to
return invocation and plan infos. The leak in byte arrays contained is taking
more space.
A leak rate of 14 ReturnInfo instances per compile was seen a different
instance. In that case compiling the statement 4000 times would have lead to
the 256 MB JVM heap being exhausted. This can result in errors for other users
of the JVM such as metadata queries or repository statements.
public static class ReturnInfo
{
// return status:
// <0: Internal error, check for Java exception
// 0: Success
// >0: User-generated error, returnedSQLState_ and
// returnedErrorMessage_ have details
int returnStatus_;
String returnedSQLState_;
String returnedMessage_;
byte [] returnedInvocationInfo_;
byte [] returnedPlanInfo_;
...
}
The fix is remove static keyword here. I could not think of a good reason on
why this class needed to be static.I went with the assumption that if possible
it is better for a nested class to be non-static as GC cleanup of non-static
classes is easier to visualize (same as other non-static classes). Also added a
deleteLocalRef where the ReturnInfo is instantiated on the C++ JNI side. It was
delete localRef that caused the leak to go away. It is not clear if the change
to non-static is necessary. I am going with the assumption that non-static is
better here.
Heap analysis commands used before now show no live instances of ReturnInfo.
Byte array size also does not grow with increasing number of compiles.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
