[
https://issues.apache.org/jira/browse/NUTCH-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13187874#comment-13187874
]
Edward Drapkin commented on NUTCH-1201:
---------------------------------------
Does this still need to be done? It seems pretty easy and I'll volunteer to do
it if it needs to be done.
I was thinking of breaking all of Fetcher apart into more easily
compartmentalized and pluggable units for my own benefit (as right now it's an
enormous class that's extremely daunting and hard to change). If this issue
still needs work, I think I can break Fetcher apart to allow for pluggable
fetchers, item queues, queue feeders and fetcher threads, but I don't want to
invest time into reinventing a wheel that you've already invented (and may not
have updated JIRA).
Let me know!
> Allow for different FetcherThread impls
> ---------------------------------------
>
> Key: NUTCH-1201
> URL: https://issues.apache.org/jira/browse/NUTCH-1201
> Project: Nutch
> Issue Type: New Feature
> Components: fetcher
> Reporter: Markus Jelsma
> Assignee: Markus Jelsma
> Fix For: 1.5
>
>
> For certain cases we need to modify parts in FetcherThread and make it
> pluggable. This introduces a new config directive fetcher.impl that takes a
> FQCN and uses that setting Fetcher.fetch to load a class to use for
> job.setMapRunnerClass(). This new class has to extend Fetcher and and inner
> class FetcherThread. This allows for overriding methods in FetcherThread but
> also methods in Fetcher itself if required.
> A follow up on this issue would be to refactor parts of FetcherThread to make
> it easier to override small sections instead of copying the entire method
> body for a small change, which is now the case.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
