Yes，this is better. I can fix the patch to apply this config. -- Best regards,
Ivy Tang On 2013-7-21, at 上午1:02, Eric Yang (JIRA) <[email protected]> wrote: > > [ > https://issues.apache.org/jira/browse/CHUKWA-581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13714480#comment-13714480 > ] > > Eric Yang commented on CHUKWA-581: > ---------------------------------- > > How about make it more backward compatible with some tweaks to config? > > {code} > <property> > <name>DataType1</name> > <value>mapperClass,reducerClass</value> > </property> > > <property> > <name>mapperType1</name> > <value>mapperClass</value> > </property> > > <property> > <name>ReduceType1</name> > <value>,reducerClass</value> > </property> > {code} > >> Demux do not support user custom reducer package name. >> ------------------------------------------------------- >> >> Key: CHUKWA-581 >> URL: https://issues.apache.org/jira/browse/CHUKWA-581 >> Project: Chukwa >> Issue Type: Improvement >> Components: MR Data Processors >> Affects Versions: 0.4.0, 0.5.0, 0.6.0 >> Environment: Ubuntu 10.0.4,Hadoop 0.20 >> Reporter: Yingzhong Xu >> Assignee: IvyTang >> Priority: Critical >> Labels: patch >> Fix For: 0.5.0, 0.6.0 >> >> Attachments: custom_reducer_conf.patch, custom_reducer.patch >> >> Original Estimate: 24h >> Remaining Estimate: 24h >> >> In ReduceProcessorFactory.java, >> it only gets {reduceType}.class from this package,as following: >> bq. String path = >> "org.apache.hadoop.chukwa.extraction.demux.processor.reducer." + reduceType; >> So our custom reducer could only be placed in that package. >> Thus, When we add our {demux}.jar to Hadoop classpath, since the Hadoop >> framework will not be able to merge the class files from multiple jar files >> with the same package name, custom reduce class can not be found. >> I get a way to overcome this defect.Put custom reducer into a package named >> with "org.apache.hadoop.chukwa.extraction.demux.processor.reducer."+{custom} >> Here,{custom} is a part of package name.Then in parser,we can write like >> this: >> bq. Key.setReducerType("{custom}.YourReducer") >> In this way, Hadoop framework could be able to find our reduce class in this >> partial user defined new package. >> But I think it will be better to fix chukwa to remove the default package >> name and fully support custom package name for reducer. >> Thanks for Eric Yang's help. > > -- > This message is automatically generated by JIRA. > If you think it was sent incorrectly, please contact your JIRA administrators > For more information on JIRA, see: http://www.atlassian.com/software/jira
