[
https://issues.apache.org/jira/browse/CHUKWA-581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13728734#comment-13728734
]
Eric Yang commented on CHUKWA-581:
----------------------------------
The test case is failing for TestDemuxMapperConfigs:
Running org.apache.hadoop.chukwa.extraction.demux.TestDemuxMapperConfigs
Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.343 sec <<<
FAILURE!
Results :
Failed tests:
testSeCustomeMapProcessor(org.apache.hadoop.chukwa.extraction.demux.TestDemuxMapperConfigs):
MockMapProcessor never invoked
Tests run: 2, Failures: 1, Errors: 0, Skipped: 0
Stacktrace:
-------------------------------------------------------------------------------
Test set: org.apache.hadoop.chukwa.extraction.demux.TestDemuxMapperConfigs
-------------------------------------------------------------------------------
Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.343 sec <<<
FAILURE!
testSeCustomeMapProcessor(org.apache.hadoop.chukwa.extraction.demux.TestDemuxMapperConfigs)
Time elapsed: 0.281 sec <<< FAILURE!
junit.framework.AssertionFailedError: MockMapProcessor never invoked
at junit.framework.Assert.fail(Assert.java:50)
at junit.framework.Assert.assertTrue(Assert.java:20)
at junit.framework.Assert.assertNotNull(Assert.java:218)
at
org.apache.hadoop.chukwa.extraction.demux.TestDemuxMapperConfigs.testSeCustomeMapProcessor(TestDemuxMapperConfigs.java:86)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at
org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at
org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
at
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
at
org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:175)
at
org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:81)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:68)
> Demux do not support user custom reducer package name.
> -------------------------------------------------------
>
> Key: CHUKWA-581
> URL: https://issues.apache.org/jira/browse/CHUKWA-581
> Project: Chukwa
> Issue Type: Improvement
> Components: MR Data Processors
> Affects Versions: 0.4.0, 0.5.0, 0.6.0
> Environment: Ubuntu 10.0.4,Hadoop 0.20
> Reporter: Yingzhong Xu
> Assignee: IvyTang
> Priority: Critical
> Labels: patch
> Fix For: 0.5.0, 0.6.0
>
> Attachments: custom_reducer_backwardcompatible.patch,
> custom_reducer_conf_backwardcompatible.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> In ReduceProcessorFactory.java,
> it only gets {reduceType}.class from this package,as following:
> bq. String path =
> "org.apache.hadoop.chukwa.extraction.demux.processor.reducer." + reduceType;
> So our custom reducer could only be placed in that package.
> Thus, When we add our {demux}.jar to Hadoop classpath, since the Hadoop
> framework will not be able to merge the class files from multiple jar files
> with the same package name, custom reduce class can not be found.
> I get a way to overcome this defect.Put custom reducer into a package named
> with "org.apache.hadoop.chukwa.extraction.demux.processor.reducer."+{custom}
> Here,{custom} is a part of package name.Then in parser,we can write like this:
> bq. Key.setReducerType("{custom}.YourReducer")
> In this way, Hadoop framework could be able to find our reduce class in this
> partial user defined new package.
> But I think it will be better to fix chukwa to remove the default package
> name and fully support custom package name for reducer.
> Thanks for Eric Yang's help.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
