pipes combiner has a large memory footprint
-------------------------------------------
Key: HADOOP-6081
URL: https://issues.apache.org/jira/browse/HADOOP-6081
Project: Hadoop Core
Issue Type: Bug
Components: pipes
Affects Versions: 0.18.3
Reporter: Christian Kunz
Pipes combiner implementation can have a huge memory overhead compared to the
spill size. How much, depends on the record size. E.g., an application asks for
>2GB memory when io.sort.mb=500, key is 16 bytes, and value is 4 bytes.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
