Matt, AFAIK Nutch does not support fetching arbitrary fetch lists out of the box.
here is a tool in JIRA that supports this though: http://issues.apache.org/jira/browse/NUTCH-68. - Thomas On 6/25/06, Honda-Search Administrator <[EMAIL PROTECTED]> wrote:
I'm having a difficult time configuring nutch to behave the way I want it to behave. In a nutshell here is my situation: I crawl a number of forums that relate to Hondas every night for posts. The purpose of my website is to be a search engine for all of the forums at once. I have a base set of URLs in the webDB right now. Every day I write a file of URLs (that I place in urls/inject.txt) that I want nutch to inject into the database to crawl. I do NOT want to recrawl other URLS. I only want to crawl/recrawl the urls in my list. Can you help me configure nutch (or help with the correct scripts, crons, etc.) to do this? i've tried without success. I am running nutch 0.7.2 and am totally confused with what to do next. It seems to me to be a simple fix, but I can't figure it out. As I mentioned I will pay if someone can set me up. I've run the crawl a number of times now and i just keep on screwing things up. Matt
