[
https://issues.apache.org/jira/browse/MAPREDUCE-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13757262#comment-13757262
]
Raymond Liu commented on MAPREDUCE-5491:
----------------------------------------
hmm, I did try move RandomTextWriter's logic here, it yeild to about 1:4
compression rate, did you think it is acceptable? I will see is there simpler
random way to gain similar outputs.
> DFSIO do not initialize write buffer correctly
> ----------------------------------------------
>
> Key: MAPREDUCE-5491
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5491
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: benchmarks, test
> Affects Versions: trunk
> Reporter: Raymond Liu
> Assignee: Raymond Liu
> Attachments: MAPREDUCE-5491.patch
>
>
> In DFSIO test, the IOMapperBase will set bufferSize in configure method,
> while writeMapper, appendMapper etc use bufferSize to initialize buffer in
> the constructor. This will lead to buffer not initialized at all. It is ok
> for non compression route, while compression is used, the output data size
> will be very small due to all 0 in buffer.
> Thus, the overrided configure method should be be the correct place for
> initial buffer
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira