[
https://issues.apache.org/jira/browse/TEZ-2732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14707744#comment-14707744
]
Rajesh Balamohan commented on TEZ-2732:
---------------------------------------
committed to branch-0.6 and 0.5
>>
branch 0.6: 253e8dee424d400495804794329f3670d8439fe0
branch 0.5: 92c84a381a91de19be14db757291a7d89eecdfef
>>
> DefaultSorter throws ArrayIndex exceptions on 2047 Mb size sort buffers
> -----------------------------------------------------------------------
>
> Key: TEZ-2732
> URL: https://issues.apache.org/jira/browse/TEZ-2732
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Fix For: 0.7.1
>
> Attachments: TEZ-2732.1.patch, TEZ-2732.branch-0.6-and-0.5.patch,
> TEZ-2732.branch-0.7.patch
>
>
> {noformat}
> kvbuffer.length = 2146435072 (2047 MB)
> Corner case: bufIndex=2026133899, kvbidx=523629312.
> distkvi = mod - i + j = 2146435072 - 2026133899 + 523629312 = 643930485
> newPos = (2026133899 + (max(.., min(643930485/2, 271128624))) (This would
> overflow)
> {noformat}
> Would be good to restrict the max allowed sort buffer to 1800 instead of
> 2047.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
