----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27169/#review59558 -----------------------------------------------------------
Ship it! Ship It! - Daniel Dai On Nov. 2, 2014, 1:24 p.m., Rohini Palaniswamy wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/27169/ > ----------------------------------------------------------- > > (Updated Nov. 2, 2014, 1:24 p.m.) > > > Review request for pig. > > > Bugs: PIG-3979 > https://issues.apache.org/jira/browse/PIG-3979 > > > Repository: pig > > > Description > ------- > > Fixed a couple of issues with POPartialAgg > - Made the spill of POPartialAgg synchronous so that System.gc() in > SpillableMemoryManager actually frees up memory. > - Avoid lot of redundant aggregateSecondLevel() calls > - Fixed the SpillableMemoryManager to not invoke extraGC if POPartialAgg > - Made variables transient which are not required to be serialized in the > plan > > > Diffs > ----- > > > http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POPartialAgg.java > 1635881 > > http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/util/GroupingSpillable.java > PRE-CREATION > > http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/util/SpillableMemoryManager.java > 1635881 > > http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestPOPartialAgg.java > 1635881 > > Diff: https://reviews.apache.org/r/27169/diff/ > > > Testing > ------- > > Unit tests added to TestPOPartialAgg. Ran couple of manual e2e tests to check > behaviour. > > > Thanks, > > Rohini Palaniswamy > >
