Re: [Nutch-dev] [bug-fix] content type

[Massimo Miccoli]

No one Can start fetch proces with lates CVS.  Ther's any patch for this problem? See my  message [Nutch-dev] Other bugs in fetcher lates version (no Http.1.1) The problem come after 10/100 url fetched. Massimo Doug Cutting wrote: [EMAIL PROTECTED] wrote: If the parsing function is moved out from current Fetcher into separate tool or tools, those issues will be less acute, if not disappearing totally. What I want to see is flexibility of being able to parse content not only by Content-Type, file name extension, but also by url pattern (and maybe other things), as demonstrated in my earlier patches. If people really want to have everything done in one pass, we can cook one tool wrapping them together. What do you think? If sounds okay, I am willing to convert current Fetcher into individual tools. That would be great.  But please make it a configurable option.  I think parsing while fetching should still be the default for now. Fetching is not CPU intensive, so parsing can easily be done at the same time, speeding the overall process.  The typical Nutch user is probably fetching predominantly HTTP/HTML sites, so we want that to be the case that works best and easiest when Nutch is first downloaded.  Does that make sense? Doug ------------------------------------------------------- This SF.Net email is sponsored by: GNOME Foundation Hackers Unite!  GUADEC: The world's #1 Open Source Desktop Event. GNOME Users and Developers European Conference, 28-30th June in Norway http://2004/guadec.org _______________________________________________ Nutch-developers mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/nutch-developers ------------------------------------------------------- This SF.Net email is sponsored by: GNOME Foundation Hackers Unite! GUADEC: The world's #1 Open Source Desktop Event. GNOME Users and Developers European Conference, 28-30th June in Norway http://2004/guadec.org _______________________________________________ Nutch-developers mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/nutch-developers