Hi Lewis Doing a quick rekkie on the current fetcher issues I can see 32 issues with > 7 of them claiming to be patched up... this kinda indicates that although > there are underlying problems with the fetcher we are currently not getting > the time to address them. It also indicates that there is quite a bit of > work to be done with the fetcher... > > There are 8 issues in trunk about the fetcher - some of them unrelated to the Fetcher (NUTCH-827 <https://issues.apache.org/jira/browse/NUTCH-827> / Nutch-1193) with most of the others being improvements (NUTCH-828<https://issues.apache.org/jira/browse/NUTCH-828>/ NUTCH-1079 <https://issues.apache.org/jira/browse/NUTCH-1079>) with possibly just a very few being real issues. I completely disagree with your statement that there are underlying problems with the fetcher and that there is quite a bit of work to do with it.
The Fetcher could be made more flexible for sure and get other improvements like any other part of the code but you cannot say it is broken or in need of urgent repair. I am also concerned about getting too radical changes to such a core part of the framework, especially when more pressing issues could be looked after instead. Having said that if someone can come up with an interesting proposal for improving the Fetcher that would be very good, I would simply suggest that we then have a separate implementation for that. Thanks Julien -- * *Open Source Solutions for Text Engineering http://digitalpebble.blogspot.com/ http://www.digitalpebble.com
