[
https://issues.apache.org/jira/browse/PIG-1544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12899162#action_12899162
]
Olga Natkovich commented on PIG-1544:
-------------------------------------
One way to do this is to only use InternalCacheBags for the bags that we are
aware off upfront. Then we can have a visitor on the plan that counts the
number of bags needed and divides memory accordingly.
> proactive-spill bags should share the memory alloted for it
> -----------------------------------------------------------
>
> Key: PIG-1544
> URL: https://issues.apache.org/jira/browse/PIG-1544
> Project: Pig
> Issue Type: Bug
> Reporter: Thejas M Nair
>
> Initially proactive spill bags were designed for use in (co)group
> (InternalCacheBag) and they knew the total number of proactive bags that were
> present, and shared the memory limit specified using the property
> pig.cachedbag.memusage .
> But the two proactive bag implementations were added later -
> InternalDistinctBag and InternalSortedBag are not aware of actual number of
> bags being used - their users always assume total-numbags = 3.
> This needs to be fixed and all proactive-spill bags should share the
> memory-limit .
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
