[
https://issues.apache.org/jira/browse/TEZ-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15246506#comment-15246506
]
Siddharth Seth commented on TEZ-3212:
-------------------------------------
[~jeagles] - this is a single key being extremely large ? I think there'll be
additional places where this may need to be looked at, and possibly move to
disk instead of trying to handle everything via memory. The sorters do that -
writing a single record out per file.
> IFile throws NegativeArraySizeException for value sizes between 1GB and 2GB
> ---------------------------------------------------------------------------
>
> Key: TEZ-3212
> URL: https://issues.apache.org/jira/browse/TEZ-3212
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Jonathan Eagles
> Assignee: Jonathan Eagles
> Attachments: TEZ-3212.1.patch
>
>
> This is not a regression with respect to MR, just an issue that was
> encountered with a job whose IFile record values (which can be of max size
> 2GB) which can be successfully written but not successfully read.
> Failure while running task:java.lang.NegativeArraySizeException
> at
> org.apache.tez.runtime.library.common.sort.impl.IFile$Reader.nextRawValue(IFile.java:765)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
