[
https://issues.apache.org/jira/browse/PIG-1348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Richard Ding updated PIG-1348:
------------------------------
Status: Resolved (was: Patch Available)
Hadoop Flags: [Reviewed]
Resolution: Fixed
The patch is committed to both trunk and 0.7 branch.
> PigStorage making unnecessary byte array copy when storing data
> ---------------------------------------------------------------
>
> Key: PIG-1348
> URL: https://issues.apache.org/jira/browse/PIG-1348
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.7.0
> Reporter: Ashutosh Chauhan
> Assignee: Richard Ding
> Fix For: 0.7.0
>
> Attachments: PIG-1348.patch, PIG-1348_2.patch
>
>
> InternalCachedBag makes estimate of memory available to the VM by using
> Runtime.getRuntime().maxMemory(). It then uses 10%(by default, though
> configurable) of this memory and divides this memory into number of bags. It
> keeps track of the memory used by bags and then proactively spills if bags
> memory usage reach close to these limits. Given all this in theory when
> presented with data more then it can handle InternalCachedBag should not run
> out of memory. But in practice we find OOM happening.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
