Anyone have the same partition problem using Streaming in 0.21.0?
On Fri, Jun 24, 2011 at 10:22 AM, steven zhuang <[email protected]>wrote: > hello, list, > > Recently I have upgraded our Hadoop cluster from 0.20.2 to > 0.21.0, and I found there is something wrong introduced by the upgrade. > > In the setConf method of > org.apache.hadoop.util.ReflectionUtils, any instance of Configurable would > be configured twice, and this may cause trouble. > For example, in 0.21.0, KeyFieldBasedPartitioner implements > the Configurable interface. When configured twice, it get two KeyDescription > and gives out wrong partition number. > > I have created a ticket for this: > https://issues.apache.org/jira/browse/HADOOP-7425 > > > Paste the source code in 0.21.0 (and 0.20.2 too) below: > > public static void setConf(Object theObject, Configuration conf) { > if (conf != null) { > if (theObject instanceof Configurable) { > ((Configurable) theObject).setConf(conf); > } > setJobConf(theObject, conf); > } > } > > > > -- > best wishes. > steven > > -- best wishes. steven
