[
https://issues.apache.org/jira/browse/TEZ-3634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siddharth Seth updated TEZ-3634:
--------------------------------
Attachment: TEZ-3634.01.patch
[~rajesh.balamohan], [~gopalv] - could you please take a look.
> reduce the buffer sizes in PipelinedSorter by a small amount
> ------------------------------------------------------------
>
> Key: TEZ-3634
> URL: https://issues.apache.org/jira/browse/TEZ-3634
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Attachments: TEZ-3634.01.patch
>
>
> When running in large JVMs with the G1GC, the regionsize maxes out at 32MB.
> PipelinedSorter does allocations of 32MB and 256MB. The actual allocation
> ends up being 24 bytes over 32M/256M. As a result the 32M allocation ends up
> taking 2 regions (or 64M), and the 256M ends up taking 9 regions instead of
> 8. Also leads to fragmentation of the heap, resulting in more Full GCs than
> required.
> {code}
> Live humongous region 2 size 33554456 start 0x00007f4918000000 length 2 with
> remset 0 code roots 0 is marked 0 reclaim candidate 0 type array 1
> {code}
> {code}
> Dead humongous region 111 size 268435480 start 0x00007f49f2000000 length 9
> with remset 0 code roots 0 is marked 0 reclaim candidate 1 type array 1
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
