[
https://issues.apache.org/jira/browse/MAPREDUCE-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220701#comment-13220701
]
Amar Kamat commented on MAPREDUCE-2722:
---------------------------------------
Ravi, compression-emulation is a feature having 3 parts
# Input compression emulation
# Intermediate compression emulation
# Output compression emulation
Intermediate and output compression emulation happens only when the
compression-emulation feature is turned on and the job's config has those
parameters set.
For input compression, Gridmix relies on 'mapred.input.dir'. If there are
compressed input files only then input compression emulation will be attempted.
Scale the input-data-size field only if input-compression-emulation is desired.
> Gridmix simulated job's map's hdfsBytesRead counter is wrong when compressed
> input is used
> ------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2722
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2722
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/gridmix
> Reporter: Ravi Gummadi
> Assignee: Ravi Gummadi
> Attachments: 2722.v1.patch, MR2722.patch
>
>
> When compressed input was used by original job's map task, then the simulated
> job's map task's hdfsBytesRead counter is wrong if compression emulation is
> enabled. This issue is because hdfsBytesRead of map task of original job is
> considered as uncompressed map input size by Gridmix.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
