[
https://issues.apache.org/jira/browse/HIVE-15221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15931075#comment-15931075
]
Fei Hui commented on HIVE-15221:
--------------------------------
Writing a sample code, prove that gc happens when we call system.gc.
JDK is openjdk version "1.8.0_91".
The result is in stat_gc.png, uploaded
{code:title=TestSystemGC.java|borderStyle=solid}
public class TestSystemGC {
public static void main(String[] args) throws Exception {
int gcCount = 0;
while(gcCount <= 10000) {
System.gc();
gcCount += 1;
System.out.println("gcCount:" + gcCount);
Thread.sleep(10000);
}
}
}
{code}
> Improvement for MapJoin checkMemoryStatus, adding gc before throwing Exception
> ------------------------------------------------------------------------------
>
> Key: HIVE-15221
> URL: https://issues.apache.org/jira/browse/HIVE-15221
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 2.1.0, 2.0.1
> Reporter: Fei Hui
> Assignee: Fei Hui
> Attachments: HIVE-15221.1.patch, stat_gc.png
>
>
> i see in the current master version
> percentage = (double) usedMemory / (double) maxHeapSize;
> if percentage > maxMemoryUsage, then throw MapJoinMemoryExhaustionException
> in my opinion, running is better than fail. after System.gc, ' if percentage
> > maxMemoryUsage, then throw MapJoinMemoryExhaustionException' maybe better
> And original checking way has a problem: 1) consuming much memory cause gc
> (e.g young gc), then check after adding row and pass. 2) consuming much
> memory does not cause gc, then check after adding rows but throw Exception
> sometimes 2) occurs, but it contians less rows than 1).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
