Hi,Eric In order to applying to all multiple datatypes in Demux,init the Reducer class depends on the ChukwaRecordKey's datatype . I need the datatype in ChukwaRecordKey.
The ChukwaRecordKey needs to add a field named dataType , showing this ChukwaRecord's datatype . And also i need add an argument in parse(String recordEntry, OutputCollector<ChukwaRecordKey, ChukwaRecord> output, Reporter reporter) An argument String dataType needs to be passed to this method to build the ChukwaRecordKey. Adding a field in ChukwaRecordKey ,and add an argument in method AbstractProcessor.parse(String recordEntry, OutputCollector<ChukwaRecordKey, ChukwaRecord> output, Reporter reporter) Is that acceptable? This is the only way i can think of to make the custom reducer more configurable. -- Best regards, Ivy Tang On 2013-7-10, at 下午12:01, Eric Yang <[email protected]> wrote: > Yes that will help. Please include test case as well. Thanks Ivy. > > regards, > Eric > > > On Mon, Jul 8, 2013 at 10:42 PM, IvyTang <[email protected]> wrote: > >> This issue has been reported >> https://issues.apache.org/jira/browse/CHUKWA-581. >> >> And i think i can submit a patch later . >> >> Should i open a new jira or just assign this CHUKWA-581 to myself? >> >> This can implement reducer outside chukwa. >> >> -- >> Best regards, >> >> Ivy Tang >> >> On 2013-7-9, at 下午1:22, Eric Yang <[email protected]> wrote: >> >>> The recommended change can improve usability to implement parsers outside >>> of Chukwa packages. +1 on the change. Do you mind open a jira for this? >>> Thanks >>> >>> regards, >>> Eric >>> >>> >>> On Mon, Jul 8, 2013 at 6:54 AM, 唐 颖 <[email protected]> wrote: >>> >>>> We can write our own mapper ,because MapProcessorFactory uses >>>> Class.forName(parserClass).getConstructor().newInstance(); >>>> since we config the parserClass in config file. >>>> >>>> But the ReduceProcessorFactory inits the reducer class using the >>>> classname >> "org.apache.hadoop.chukwa.extraction.demux.processor.reducer."+ >>>> reduceType. >>>> So the reducer class must be put in package >>>> org.apache.hadoop.chukwa.extraction.demux.processor.reducer and named >>>> reduceType. >>>> >>>> Why not we put the reducer class name in config file either? >>>> For exam, the {customdatatype_mapper} means mapper class >>>> ,{customdatatype_reducer} means reducer class. >>>> >>>> This can give chukka more flexibility. >>>> >>>> -- >>>> Best regards, >>>> >>>> Ivy Tang >>>> >>>> >> >>
