[
https://issues.apache.org/jira/browse/TEZ-2085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317738#comment-14317738
]
Hitesh Shah commented on TEZ-2085:
----------------------------------
{code}
if (bufferOverflowCount > bufferList.size()) {
272 throw new MapBufferTooSmallException("Record too large for
in-memory buffer. Exceeded "
273 + "buffer overflow limit, bufferOverflowCount=" +
bufferOverflowCount + ", bufferList"
274 + ".size=" + bufferList.size() + ", blockSize=" +
blockSize);
275 }
{code}
Should this mention what config property should be modified?
> PipelinedSorter should bail out (on BufferOverflowException) instead of
> retrying continuously
> ---------------------------------------------------------------------------------------------
>
> Key: TEZ-2085
> URL: https://issues.apache.org/jira/browse/TEZ-2085
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Attachments: TEZ-2085.1.patch, TEZ-2085.2.patch
>
>
> If we try to fit in a key/value pair which is great than the size that sort
> span can accommodate, PipelinedSorter would try to sort/spill indefinitely.
> This is more of a corner case. It should bail out gracefully and can throw
> back IOException instead.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
