[
https://issues.apache.org/jira/browse/TEZ-2085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318983#comment-14318983
]
Gopal V commented on TEZ-2085:
------------------------------
Looks good, I suggest making all the PipelinedSorter parameters which do not
change during execution as final parameters.
blockSize & capacity are clearly not modifiable once the sorter kicks off, so
make them final vars at the top, so that we can debug through this without
mental branch-mispredictions.
The recursive counter needs a comment to indicate that it is not the total
number of bufferOverflows, but it is a recursive depth safety check - perhaps
name it bufferOverflowRecursion instead?
> PipelinedSorter should bail out (on BufferOverflowException) instead of
> retrying continuously
> ---------------------------------------------------------------------------------------------
>
> Key: TEZ-2085
> URL: https://issues.apache.org/jira/browse/TEZ-2085
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Attachments: TEZ-2085.1.patch, TEZ-2085.2.patch, TEZ-2085.3.patch
>
>
> If we try to fit in a key/value pair which is great than the size that sort
> span can accommodate, PipelinedSorter would try to sort/spill indefinitely.
> This is more of a corner case. It should bail out gracefully and can throw
> back IOException instead.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
