[
https://issues.apache.org/jira/browse/PIG-4770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohini Palaniswamy updated PIG-4770:
------------------------------------
Attachment: PIG-4770-1.patch
Changes done:
1) Avoid hashmap resizing and arraylist (value of hashmap) resizing as
much as possible to reduce the memory pressure
- Optimize group all by initializing the values List to same size
as maxListSize.
- Changed default minReduction to 7 as secondary hashmap will grow to
4096 with default reduction of 10 or 7. 7x reduction is also good for
performance. So lowered the default.
2) Fixed an NPE in SpillableMemoryManager which was due to not using
precalculated memory size. It was a mistake in my fix for PIG-4012.
> OOM with POPartialAgg in some cases
> -----------------------------------
>
> Key: PIG-4770
> URL: https://issues.apache.org/jira/browse/PIG-4770
> Project: Pig
> Issue Type: Bug
> Reporter: Rohini Palaniswamy
> Assignee: Rohini Palaniswamy
> Fix For: 0.16.0
>
> Attachments: PIG-4770-1.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
