[
https://issues.apache.org/jira/browse/PIG-3325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689465#comment-13689465
]
Dmitriy V. Ryaboy commented on PIG-3325:
----------------------------------------
What if instead of figuring out size based on the first 100 elements, we
sampled first, 11th, 21st, etc until we get 100 samples? Would help with small
bags (where accuracy of estimate doesn't matter as much).
> Adding a tuple to a bag is slow
> -------------------------------
>
> Key: PIG-3325
> URL: https://issues.apache.org/jira/browse/PIG-3325
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.11, 0.11.1, 0.11.2
> Reporter: Mark Wagner
> Assignee: Mark Wagner
> Priority: Critical
> Attachments: PIG-3325.demo.patch, PIG-3325.optimize.1.patch
>
>
> The time it takes to add a tuple to a bag has increased significantly,
> causing some jobs to take about 50x longer compared to 0.10.1. I've tracked
> this down to PIG-2923, which has made adding a tuple heavier weight (it now
> includes some memory estimation).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
