wang qiang created TEZ-4133:
-------------------------------
Summary: key class implments writableComparable and configurable
use default configuration
Key: TEZ-4133
URL: https://issues.apache.org/jira/browse/TEZ-4133
Project: Apache Tez
Issue Type: Bug
Reporter: wang qiang
key class implments writableComparable and configurable ,setConf be invoked
with a default configuration,it should like
mapreduce(org.apache.hadoop.mapred.JobConf#getOutputKeyComparator)
{code:java}
//代码占位符
public static <K> RawComparator<K>
getIntermediateOutputKeyComparator(Configuration conf) {
Class<? extends RawComparator> theClass = conf.getClass(
TezRuntimeConfiguration.TEZ_RUNTIME_KEY_COMPARATOR_CLASS, null,
RawComparator.class);
if (theClass != null)
return ReflectionUtils.newInstance(theClass, conf);
// aybe change to
// return
WritableComparator.get(getIntermediateOutputKeyClass(conf).asSubclass(
//WritableComparable.class),conf);
return WritableComparator.get(getIntermediateOutputKeyClass(conf).asSubclass(
WritableComparable.class));
}
public static <K> RawComparator<K>
getIntermediateInputKeyComparator(Configuration conf) {
Class<? extends RawComparator> theClass = conf.getClass(
TezRuntimeConfiguration.TEZ_RUNTIME_KEY_COMPARATOR_CLASS, null,
RawComparator.class);
if (theClass != null)
return ReflectionUtils.newInstance(theClass, conf);
// maybe change to
// return WritableComparator.get(getIntermediateInputKeyClass(conf).asSubclass(
//WritableComparable.class),conf);
return WritableComparator.get(getIntermediateInputKeyClass(conf).asSubclass(
WritableComparable.class));
}
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
