[
https://issues.apache.org/jira/browse/PIG-3325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitriy V. Ryaboy updated PIG-3325:
-----------------------------------
Attachment: PIG-3325.2.patch
Updating with a patch.
Results:
||Num Tuples in Bag || Trunk avg || Patch 1 avg || Patch 2 avg ||
| 1 | round: 0.00 | round: 0.00 | round: 0.00 |
| 20 | round: 0.01 | round: 0.00 | round: 0.00 |
| 100 | round: 0.13 | round: 0.00 | round: 0.00
| 1000 | round: 0.19 | round: 1.20 | round: 0.03 |
I also ran Mark's bench test in a loop 10 times (again, to account for jit
effects).
Results are as follows:
My Patch, Mark's test
7050 ns
450 ns
440 ns
550 ns
440 ns
440 ns
440 ns
440 ns
440 ns
540 ns
410 ns
440 ns
440 ns
430 ns
460 ns
Trunk, Mark's test
243240 ns
156640 ns
25440 ns
23470 ns
18930 ns
20710 ns
16890 ns
20210 ns
17630 ns
17900 ns
21420 ns
22550 ns
22900 ns
19800 ns
16770 ns
Mark's patch, Mark's Test
8480 ns
2750 ns
2690 ns
2760 ns
3270 ns
3590 ns
6530 ns
5900 ns
6340 ns
5410 ns
5400 ns
5420 ns
5670 ns
5410 ns
5420 ns
> Adding a tuple to a bag is slow
> -------------------------------
>
> Key: PIG-3325
> URL: https://issues.apache.org/jira/browse/PIG-3325
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.11, 0.11.1, 0.11.2
> Reporter: Mark Wagner
> Assignee: Mark Wagner
> Priority: Critical
> Attachments: PIG-3325.2.patch, PIG-3325.demo.patch,
> PIG-3325.optimize.1.patch
>
>
> The time it takes to add a tuple to a bag has increased significantly,
> causing some jobs to take about 50x longer compared to 0.10.1. I've tracked
> this down to PIG-2923, which has made adding a tuple heavier weight (it now
> includes some memory estimation).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
