[
https://issues.apache.org/jira/browse/CHUKWA-581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Yang updated CHUKWA-581:
-----------------------------
Fix Version/s: (was: 0.4.0)
> Demux do not support user custom reducer package name.
> -------------------------------------------------------
>
> Key: CHUKWA-581
> URL: https://issues.apache.org/jira/browse/CHUKWA-581
> Project: Chukwa
> Issue Type: Improvement
> Components: MR Data Processors
> Affects Versions: 0.4.0
> Environment: Ubuntu 10.0.4,Hadoop 0.20
> Reporter: Yingzhong Xu
> Priority: Critical
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> In ReduceProcessorFactory.java,
> it only gets {reduceType}.class from this package,as following:
> bq. String path =
> "org.apache.hadoop.chukwa.extraction.demux.processor.reducer." + reduceType;
> So our custom reducer could only be placed in that package.
> Thus, When we add our {demux}.jar to Hadoop classpath, since the Hadoop
> framework will not be able to merge the class files from multiple jar files
> with the same package name, custom reduce class can not be found.
> I get a way to overcome this defect.Put custom reducer into a package named
> with "org.apache.hadoop.chukwa.extraction.demux.processor.reducer."+{custom}
> Here,{custom} is a part of package name.Then in parser,we can write like this:
> bq. Key.setReducerType("{custom}.YourReducer")
> In this way, Hadoop framework could be able to find our reduce class in this
> partial user defined new package.
> But I think it will be better to fix chukwa to remove the default package
> name and fully support custom package name for reducer.
> Thanks for Eric Yang's help.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
