[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13757322#comment-13757322
 ] 

Todd Lipcon commented on MAPREDUCE-5491:
----------------------------------------

IMO we should move to a random array of bytes and ensure that the array is 
larger than typical compression windows -- eg allocate an 8MB buffer and do 8MB 
writes into the stream. We shouldn't have output that's in any way compressible 
for a test that's ostensibly about IO performance. (or, it should be an option)
                
> DFSIO do not initialize write buffer correctly
> ----------------------------------------------
>
>                 Key: MAPREDUCE-5491
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5491
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: benchmarks, test
>    Affects Versions: trunk
>            Reporter: Raymond Liu
>            Assignee: Raymond Liu
>         Attachments: MAPREDUCE-5491.patch
>
>
> In DFSIO test, the IOMapperBase will set bufferSize in configure method, 
> while writeMapper, appendMapper etc use bufferSize to initialize buffer in 
> the constructor. This will lead to buffer not initialized at all. It is ok 
> for non compression route, while compression is used, the output data size 
> will be very small due to all 0 in buffer.
> Thus, the overrided configure method should be be the correct place for 
> initial buffer

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Reply via email to