[ https://issues.apache.org/jira/browse/PIG-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-1307: ---------------------------- Resolution: Fixed Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available) Performed a manual test, and it works. Patch committed. Thanks Ben! > when we spill the DefaultDataBag we are not setting the sized changed flag to > be true. > -------------------------------------------------------------------------------------- > > Key: PIG-1307 > URL: https://issues.apache.org/jira/browse/PIG-1307 > Project: Pig > Issue Type: Bug > Reporter: Benjamin Reed > Assignee: Benjamin Reed > Fix For: 0.7.0 > > Attachments: PIG-1307.patch > > > pig uses a size changed flag to indicate when we should recalculate the > memory footprint of the bag. the setting of this flag is sprinkled throughout > the code. unfortunately, it is missing in DefaultDataBag.spill(). there may > be other cases as well. the problem with this case is that when the low > memory threshold kicks in, bags are spilled until the desired amount of > memory is "freed". since the flag is not being reset subsequent calls to the > threshold events will retrigger the spill() and think more memory was freed > even though nothing was actually spilled. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.