-----Original message----- > From:Bayu Widyasanyata <[email protected]> > Sent: Sun 13-Jan-2013 07:34 > To: [email protected] > Subject: Re: How segments is created? > > On Sun, Jan 13, 2013 at 12:47 PM, Tejas Patil <[email protected]>wrote: > > > > > Well, if you know that the front page is updated frequently, set > > "db.fetch.interval.default" to lower value so that urls will be eligible > > for re-fetch sooner. By default, if a url is fetched successfully, it > > becomes eligible for re-fetching after 30 days > > > Very clear! > In summary, > Nutch can not identify if a page is being updated hence (if page is updated > frequently) we should set to lower value "db.fetch.interval.default" to > re-fetch the page.
No, you can plugin another FetchSchedule that supports adjusting the interval based on whether a record is modified. See the AdaptiveFetchSchedule for an example. > > Thanks so much! > -- > wassalam, > [bayu] >
