Author: knoguchi
Date: Wed Jul 3 17:53:20 2019
New Revision: 1862506
URL: http://svn.apache.org/viewvc?rev=1862506&view=rev
Log:
PIG-5385: Skip calling extra gc() before spilling large bag when unnecessary
(knoguchi)
Modified:
pig/trunk/CHANGES.txt
pig/trunk/src/org/apache/pig/impl/util/SpillableMemoryManager.java
Modified: pig/trunk/CHANGES.txt
URL:
http://svn.apache.org/viewvc/pig/trunk/CHANGES.txt?rev=1862506&r1=1862505&r2=1862506&view=diff
==============================================================================
--- pig/trunk/CHANGES.txt (original)
+++ pig/trunk/CHANGES.txt Wed Jul 3 17:53:20 2019
@@ -26,6 +26,8 @@ PIG-5282: Upgade to Java 8 (satishsaley
IMPROVEMENTS
+PIG-5385: Skip calling extra gc() before spilling large bag when unnecessary
(knoguchi)
+
PIG-5389: Passing null to REPLACE udf preventing JIT optimization (knoguchi)
PIG-5382: Log the name when POStore fails on write (knoguchi)
Modified: pig/trunk/src/org/apache/pig/impl/util/SpillableMemoryManager.java
URL:
http://svn.apache.org/viewvc/pig/trunk/src/org/apache/pig/impl/util/SpillableMemoryManager.java?rev=1862506&r1=1862505&r2=1862506&view=diff
==============================================================================
--- pig/trunk/src/org/apache/pig/impl/util/SpillableMemoryManager.java
(original)
+++ pig/trunk/src/org/apache/pig/impl/util/SpillableMemoryManager.java Wed Jul
3 17:53:20 2019
@@ -293,7 +293,8 @@ public class SpillableMemoryManager impl
// Do not invoke extraGC for GroupingSpillable. Its size
will always exceed
// extraGCSpillSizeThreshold and the data is always strong
referenced.
if( !extraGCCalled && extraGCSpillSizeThreshold != 0
- && toBeFreed > extraGCSpillSizeThreshold &&
!isGroupingSpillable) {
+ && toBeFreed > extraGCSpillSizeThreshold &&
!isGroupingSpillable
+ &&
n.getType().equals(MemoryNotificationInfo.MEMORY_THRESHOLD_EXCEEDED)) {
log.debug("Single spillable has size " + toBeFreed +
"bytes. Calling extra gc()");
// this extra assignment to null is needed so that gc
can free the
// spillable if nothing else is pointing at it
