[
https://issues.apache.org/jira/browse/MAPREDUCE-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13757322#comment-13757322
]
Todd Lipcon commented on MAPREDUCE-5491:
----------------------------------------
IMO we should move to a random array of bytes and ensure that the array is
larger than typical compression windows -- eg allocate an 8MB buffer and do 8MB
writes into the stream. We shouldn't have output that's in any way compressible
for a test that's ostensibly about IO performance. (or, it should be an option)
> DFSIO do not initialize write buffer correctly
> ----------------------------------------------
>
> Key: MAPREDUCE-5491
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5491
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: benchmarks, test
> Affects Versions: trunk
> Reporter: Raymond Liu
> Assignee: Raymond Liu
> Attachments: MAPREDUCE-5491.patch
>
>
> In DFSIO test, the IOMapperBase will set bufferSize in configure method,
> while writeMapper, appendMapper etc use bufferSize to initialize buffer in
> the constructor. This will lead to buffer not initialized at all. It is ok
> for non compression route, while compression is used, the output data size
> will be very small due to all 0 in buffer.
> Thus, the overrided configure method should be be the correct place for
> initial buffer
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira