[
https://issues.apache.org/jira/browse/HADOOP-1851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12525535
]
drboscolo edited comment on HADOOP-1851 at 9/6/07 2:30 PM:
----------------------------------------------------------
The methods that affect this behavior are in line 341-371 of JobConf in Hadoop
0.14.1:
/**
\* Set the given class as the compression codec for the map outputs.
\* @param codecClass the CompressionCodec class that will compress the
\* map outputs
*/
public void setMapOutputCompressorClass(Class<? extends CompressionCodec>
codecClass) {
setCompressMapOutput(true);
setClass("mapred.output.compression.codec", codecClass,
CompressionCodec.class);
}
/**
\* Get the codec for compressing the map outputs
\* @param defaultValue the value to return if it is not set
\* @return the CompressionCodec class that should be used to compress the
\* map outputs
\* @throws IllegalArgumentException if the class was specified, but not found
*/
public Class<? extends CompressionCodec> getMapOutputCompressorClass(Class<?
extends CompressionCodec> defaultValue) {
String name = get("mapred.output.compression.codec");
if (name == null) {
return defaultValue;
} else {
try {
return getClassByName(name).asSubclass(CompressionCodec.class);
} catch (ClassNotFoundException e) {
throw new IllegalArgumentException("Compression codec " + name +
" was not found.", e);
}
}
}
This could be easily fixed by using a different property, for example,
"map.output.compression.codec".
was (Author: drboscolo):
The methods that affect this behavior are in line 341-371 of JobConf in
Hadoop 0.14.1:
/**
* Set the given class as the compression codec for the map outputs.
* @param codecClass the CompressionCodec class that will compress the
* map outputs
*/
public void setMapOutputCompressorClass(Class<? extends CompressionCodec>
codecClass) {
setCompressMapOutput(true);
setClass("mapred.output.compression.codec", codecClass,
CompressionCodec.class);
}
/**
* Get the codec for compressing the map outputs
* @param defaultValue the value to return if it is not set
* @return the CompressionCodec class that should be used to compress the
* map outputs
* @throws IllegalArgumentException if the class was specified, but not found
*/
public Class<? extends CompressionCodec> getMapOutputCompressorClass(Class<?
extends CompressionCodec> defaultValue) {
String name = get("mapred.output.compression.codec");
if (name == null) {
return defaultValue;
} else {
try {
return getClassByName(name).asSubclass(CompressionCodec.class);
} catch (ClassNotFoundException e) {
throw new IllegalArgumentException("Compression codec " + name +
" was not found.", e);
}
}
}
This could be easily fixed by using a different property, for example,
"map.output.compression.codec".
> Map output compression coded cannot be set independently of job output
> compression codec
> ----------------------------------------------------------------------------------------
>
> Key: HADOOP-1851
> URL: https://issues.apache.org/jira/browse/HADOOP-1851
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.14.1
> Reporter: Riccardo
>
> The property "mapred.output.compression.codec" is used when setting and
> getting the map output compression codec in JobConf, thus making it
> impossible to use a different codec for map outputs and overall job outputs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
