[
https://issues.apache.org/jira/browse/NUTCH-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Markus Jelsma updated NUTCH-1201:
---------------------------------
Attachment: CustomFetcher.java
NUTCH-1201-1.5-wip.patch
Here's a WIP that allows us to implement different parts of the fetcher. We use
it to for two different impl's of FetcherThread depending on customer.
I also attached a CustomFetcher, this behaves the same as the current Fetcher
and just acts as an example (to myself).
It's ugly but does the trick ;)
> Allow for different FetcherThread impls
> ---------------------------------------
>
> Key: NUTCH-1201
> URL: https://issues.apache.org/jira/browse/NUTCH-1201
> Project: Nutch
> Issue Type: New Feature
> Components: fetcher
> Reporter: Markus Jelsma
> Assignee: Markus Jelsma
> Fix For: 1.5
>
> Attachments: CustomFetcher.java, NUTCH-1201-1.5-wip.patch
>
>
> For certain cases we need to modify parts in FetcherThread and make it
> pluggable. This introduces a new config directive fetcher.impl that takes a
> FQCN and uses that setting Fetcher.fetch to load a class to use for
> job.setMapRunnerClass(). This new class has to extend Fetcher and and inner
> class FetcherThread. This allows for overriding methods in FetcherThread but
> also methods in Fetcher itself if required.
> A follow up on this issue would be to refactor parts of FetcherThread to make
> it easier to override small sections instead of copying the entire method
> body for a small change, which is now the case.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
