[
https://issues.apache.org/jira/browse/CHUKWA-581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13743118#comment-13743118
]
Hudson commented on CHUKWA-581:
-------------------------------
FAILURE: Integrated in Chukwa-trunk #470 (See
[https://builds.apache.org/job/Chukwa-trunk/470/])
CHUKWA-581. Added support for custom reducer package name. (Ivy Tang via Eric
Yang) (eyang:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1515060)
* /incubator/chukwa/trunk/CHANGES.txt
* /incubator/chukwa/trunk/conf/chukwa-demux-conf.xml
*
/incubator/chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/extraction/demux/Demux.java
*
/incubator/chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/extraction/demux/processor/reducer/ReduceProcessorFactory.java
*
/incubator/chukwa/trunk/src/test/java/org/apache/hadoop/chukwa/extraction/demux/TestDemuxMapperConfigs.java
*
/incubator/chukwa/trunk/src/test/java/org/apache/hadoop/chukwa/extraction/demux/TestDemuxReducerConfigs.java
> Demux do not support user custom reducer package name.
> -------------------------------------------------------
>
> Key: CHUKWA-581
> URL: https://issues.apache.org/jira/browse/CHUKWA-581
> Project: Chukwa
> Issue Type: Improvement
> Components: MR Data Processors
> Affects Versions: 0.4.0, 0.5.0, 0.6.0
> Environment: Ubuntu 10.0.4,Hadoop 0.20
> Reporter: Yingzhong Xu
> Assignee: IvyTang
> Priority: Critical
> Labels: patch
> Fix For: 0.6.0
>
> Attachments: custom_reducer_backwardcompatible.patch,
> custom_reducer_conf_backwardcompatible.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> In ReduceProcessorFactory.java,
> it only gets {reduceType}.class from this package,as following:
> bq. String path =
> "org.apache.hadoop.chukwa.extraction.demux.processor.reducer." + reduceType;
> So our custom reducer could only be placed in that package.
> Thus, When we add our {demux}.jar to Hadoop classpath, since the Hadoop
> framework will not be able to merge the class files from multiple jar files
> with the same package name, custom reduce class can not be found.
> I get a way to overcome this defect.Put custom reducer into a package named
> with "org.apache.hadoop.chukwa.extraction.demux.processor.reducer."+{custom}
> Here,{custom} is a part of package name.Then in parser,we can write like this:
> bq. Key.setReducerType("{custom}.YourReducer")
> In this way, Hadoop framework could be able to find our reduce class in this
> partial user defined new package.
> But I think it will be better to fix chukwa to remove the default package
> name and fully support custom package name for reducer.
> Thanks for Eric Yang's help.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
