I would say that limitation is a bug. I remember opening a Jira for
it, once. Shouldn't be too hard to fix...
sent from my iPhone; please excuse typos and brevity.
On Dec 23, 2009, at 8:36 AM, Christian <engr...@gmail.com> wrote:
Hi Eric,
This isn't an issue for me yet, but I can see it becoming one if we
end up using Chukwa (currently investigating it). What's the reason
behind requiring the package name to be org.apache.hadoop.chukwa.
extraction.demux.processor.mapper?
Regards,
Chris
On Tue, Dec 22, 2009 at 6:46 PM, Eric Yang <ey...@yahoo-inc.com>
wrote:
I thought this is the current implementation. The class file should
be in
the same package name, but it is not required to be in the same jar
file.
If it is not working, please file a jira.
Regards,
Eric
On 12/22/09 4:40 PM, "Bill Graham" <billgra...@gmail.com> wrote:
>> The extensions could be added by adding the class to the class
path of the
>> demux process. If you put your jar file in CHUKWA_HOME/lib and
update
>> chukwa-demux-conf.xml, then it should work automatically.
>
> Just to clarify, are you saying this is how it currently works or
how it could
> work in the future?
>
> Currently it doesn't work this way, which is the point of my post.
I put a jar
> in the lib/ directory of my data processor that contained my
processor. I
> mapped it in chukwa-demux.xml and bounced the data processor. I
did a ps and
> saw the jar in the DemuxManager classpath, but I still got
> ClassNotFoundExceptions.
>
> The only way I could get it to work was to do the following:
> - Move my class into
> org.apache.hadoop.chukwa.extraction.demux.processor.mapper
> - Add my compiled class to the chukwa-core jar.
>
>
> On Tue, Dec 22, 2009 at 3:11 PM, Eric Yang <ey...@yahoo-inc.com>
wrote:
>>
>>
>>
>> On 12/22/09 2:40 PM, "Bill Graham" <billgra...@gmail.com> wrote:
>>
>>> Thanks for your quick reply Eric.
>>>
>>> The TsProcessor does use buildGenericRecord and has been working
fine for me
>>> (at least I thought it was). I've mapped it to my dataType as
you described
>>> without problems. My only point with issue #1 was just that the
>>> documentation
>>> is off and that the DefaultProcessor yields what I think is
unexpected
>>> behavior.
>>>
>>
>> I will update the documentation to align with the code. Thank
you for
>> finding this.
>>
>>> Yes, annotations would be useful. Or what about just having an
extensions
>>> directory (maybe lib/ext/) or something similar where custom
jars could be
>>> placed that are to be submitted by demux M/R? Do you know where
the code
>>> resides that handles adding the chukwa-core jar? I poked around
bit but
>>> couldn't find it.
>>>
>>> Finally, is there a JIRA for this issue that you know of? If not
I'll create
>>> one. This is going to become a pain point for us soon, so if we
have a
>>> design
>>> I might be able to contribute a patch.
>>
>> The extensions could be added by adding the class to the class
path of the
>> demux process. If you put your jar file in CHUKWA_HOME/lib and
update
>> chukwa-demux-conf.xml, then it should work automatically. We
probably
>> should have a jira to document this. Please go ahead and file one.
>>
>> For your interest, the annotation jira is:
>>
>> https://issues.apache.org/jira/browse/CHUKWA-371
>>
>> Regards,
>> Eric
>>
>>
>
>
