[jira] [Created] (TEZ-2732) DefaultSorter throws ArrayIndex exceptions on 2047 Mb size sort buffers

[Rajesh Balamohan (JIRA)]

Rajesh Balamohan created TEZ-2732:
-------------------------------------

             Summary: DefaultSorter throws ArrayIndex exceptions on 2047 Mb 
size sort buffers
                 Key: TEZ-2732
                 URL: https://issues.apache.org/jira/browse/TEZ-2732
             Project: Apache Tez
          Issue Type: Bug
            Reporter: Rajesh Balamohan


{noformat}
  kvbuffer.length = 2146435072 (2047 MB)
  Corner case: bufIndex=2026133899, kvbidx=523629312.
  distkvi = mod - i + j = 2146435072 - 2026133899 + 523629312 = 643930485
  newPos = (2026133899 + (max(.., min(643930485/2, 271128624))) (This would 
overflow)
{noformat}

Would be good to restrict the max allowed sort buffer to 1800 instead of 2047. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)