Right, that is probably best for a start. It really also depends on your hardware / network setup; if you have enough resources to crawl / index your whole network in less than the default refetch interval you can consider decreading that property.
There is also work going on related to a so called adaptive refetch interval, i.e. resources that are changing frequently being refetched more often than thoes that don't. But this is not yet in trunk. Rgrds, Thomas On 6/25/06, Honda-Search Administrator <[EMAIL PROTECTED]> wrote: > Of course the other option is to treat this like any other search engine and > just let nutch "do it's thing" knowing that everything will eventually get > indexed. > > Matt > > ----- Original Message ----- > From: "TDLN" <[EMAIL PROTECTED]> > To: <[email protected]>; "Honda-Search Administrator" > <[EMAIL PROTECTED]> > Sent: Sunday, June 25, 2006 3:02 AM > Subject: Re: Will pay for someone to help > > > > Matt, > > > > AFAIK Nutch does not support fetching arbitrary fetch lists out of the > > box. > > > > here is a tool in JIRA that supports this though: > > http://issues.apache.org/jira/browse/NUTCH-68. > > > > - Thomas > > > > > > On 6/25/06, Honda-Search Administrator <[EMAIL PROTECTED]> wrote: > >> I'm having a difficult time configuring nutch to behave the way I want it > >> to > >> behave. > >> > >> In a nutshell here is my situation: > >> > >> I crawl a number of forums that relate to Hondas every night for posts. > >> The > >> purpose of my website is to be a search engine for all of the forums at > >> once. > >> > >> I have a base set of URLs in the webDB right now. Every day I write a > >> file > >> of URLs (that I place in urls/inject.txt) that I want nutch to inject > >> into > >> the database to crawl. I do NOT want to recrawl other URLS. I only want > >> to > >> crawl/recrawl the urls in my list. > >> > >> Can you help me configure nutch (or help with the correct scripts, crons, > >> etc.) to do this? i've tried without success. > >> > >> I am running nutch 0.7.2 and am totally confused with what to do next. > >> It > >> seems to me to be a simple fix, but I can't figure it out. > >> > >> As I mentioned I will pay if someone can set me up. I've run the crawl a > >> number of times now and i just keep on screwing things up. > >> > >> Matt > >> > >> > > > > > > Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Nutch-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-general
