[
https://issues.apache.org/jira/browse/TEZ-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14903821#comment-14903821
]
Gopal V commented on TEZ-2850:
------------------------------
Good catch [~saikatr], that's 4kb of space overhead for 100 bytes of data.
The perf fix was to fix the total # of JNI calls to libhadoop.so CRC32. With
this fix, the Writable deserialization is unbuffered - so an IntWritable will
trigger 1 JNI call out to libhadoop.so per 4 byte Integer read (also see
HADOOP-10778).
> Tez MergeManager OOM for small Map Outputs
> ------------------------------------------
>
> Key: TEZ-2850
> URL: https://issues.apache.org/jira/browse/TEZ-2850
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Saikat
> Attachments: OOM_1.png, OOM_2.png, OOM_3.png, TEZ-2850_test.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
