[jira] [Updated] (TEZ-4133) key class implments writableComparable and configurable use default configuration

[wang qiang (Jira)]

     [ 
https://issues.apache.org/jira/browse/TEZ-4133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

wang qiang updated TEZ-4133:
----------------------------
    Description: 
key class implments writableComparable and configurable ,setConf be invoked 
with a default configuration,it should  like 
mapreduce(org.apache.hadoop.mapred.JobConf#getOutputKeyComparator)
{code:java}
//org.apache.tez.runtime.library.common.ConfigUtils#getIntermediateOutputKeyComparator
public static <K> RawComparator<K> 
getIntermediateOutputKeyComparator(Configuration conf) {
  Class<? extends RawComparator> theClass = conf.getClass(
      TezRuntimeConfiguration.TEZ_RUNTIME_KEY_COMPARATOR_CLASS, null,
      RawComparator.class);
  if (theClass != null)
    return ReflectionUtils.newInstance(theClass, conf);
// aybe change to 
// return 
WritableComparator.get(getIntermediateOutputKeyClass(conf).asSubclass( 
//WritableComparable.class),conf);
  return WritableComparator.get(getIntermediateOutputKeyClass(conf).asSubclass(
      WritableComparable.class));
}

public static <K> RawComparator<K> 
getIntermediateInputKeyComparator(Configuration conf) {
  Class<? extends RawComparator> theClass = conf.getClass(
      TezRuntimeConfiguration.TEZ_RUNTIME_KEY_COMPARATOR_CLASS, null,
      RawComparator.class);
  if (theClass != null)
    return ReflectionUtils.newInstance(theClass, conf);
// maybe change to 
// return WritableComparator.get(getIntermediateInputKeyClass(conf).asSubclass( 
//WritableComparable.class),conf);
  return WritableComparator.get(getIntermediateInputKeyClass(conf).asSubclass(
      WritableComparable.class));
}



 {code}
 
  
  
  
  
  
  
  
  
  
  

  was:
key class implments writableComparable and configurable ,setConf be invoked 
with a default configuration,it should  like 
mapreduce(org.apache.hadoop.mapred.JobConf#getOutputKeyComparator)
{code:java}
//代码占位符
public static <K> RawComparator<K> 
getIntermediateOutputKeyComparator(Configuration conf) {
  Class<? extends RawComparator> theClass = conf.getClass(
      TezRuntimeConfiguration.TEZ_RUNTIME_KEY_COMPARATOR_CLASS, null,
      RawComparator.class);
  if (theClass != null)
    return ReflectionUtils.newInstance(theClass, conf);
// aybe change to 
// return 
WritableComparator.get(getIntermediateOutputKeyClass(conf).asSubclass( 
//WritableComparable.class),conf);
  return WritableComparator.get(getIntermediateOutputKeyClass(conf).asSubclass(
      WritableComparable.class));
}

public static <K> RawComparator<K> 
getIntermediateInputKeyComparator(Configuration conf) {
  Class<? extends RawComparator> theClass = conf.getClass(
      TezRuntimeConfiguration.TEZ_RUNTIME_KEY_COMPARATOR_CLASS, null,
      RawComparator.class);
  if (theClass != null)
    return ReflectionUtils.newInstance(theClass, conf);
// maybe change to 
// return WritableComparator.get(getIntermediateInputKeyClass(conf).asSubclass( 
//WritableComparable.class),conf);
  return WritableComparator.get(getIntermediateInputKeyClass(conf).asSubclass(
      WritableComparable.class));
}



 {code}
 
  
  
  
  
  
  
  
  
 
 


> key class implments writableComparable and configurable use default 
> configuration
> ---------------------------------------------------------------------------------
>
>                 Key: TEZ-4133
>                 URL: https://issues.apache.org/jira/browse/TEZ-4133
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: wang qiang
>            Priority: Major
>
> key class implments writableComparable and configurable ,setConf be invoked 
> with a default configuration,it should  like 
> mapreduce(org.apache.hadoop.mapred.JobConf#getOutputKeyComparator)
> {code:java}
> //org.apache.tez.runtime.library.common.ConfigUtils#getIntermediateOutputKeyComparator
> public static <K> RawComparator<K> 
> getIntermediateOutputKeyComparator(Configuration conf) {
>   Class<? extends RawComparator> theClass = conf.getClass(
>       TezRuntimeConfiguration.TEZ_RUNTIME_KEY_COMPARATOR_CLASS, null,
>       RawComparator.class);
>   if (theClass != null)
>     return ReflectionUtils.newInstance(theClass, conf);
> // aybe change to 
> // return 
> WritableComparator.get(getIntermediateOutputKeyClass(conf).asSubclass( 
> //WritableComparable.class),conf);
>   return 
> WritableComparator.get(getIntermediateOutputKeyClass(conf).asSubclass(
>       WritableComparable.class));
> }
> public static <K> RawComparator<K> 
> getIntermediateInputKeyComparator(Configuration conf) {
>   Class<? extends RawComparator> theClass = conf.getClass(
>       TezRuntimeConfiguration.TEZ_RUNTIME_KEY_COMPARATOR_CLASS, null,
>       RawComparator.class);
>   if (theClass != null)
>     return ReflectionUtils.newInstance(theClass, conf);
> // maybe change to 
> // return 
> WritableComparator.get(getIntermediateInputKeyClass(conf).asSubclass( 
> //WritableComparable.class),conf);
>   return WritableComparator.get(getIntermediateInputKeyClass(conf).asSubclass(
>       WritableComparable.class));
> }
>  {code}
>  
>   
>   
>   
>   
>   
>   
>   
>   
>   
>   



--
This message was sent by Atlassian Jira
(v8.3.4#803005)