I see that the ModifiedTime used in protocol plugins that mean if the webpage has not changed , no need to download again. And their have also used in FetchSchedule implementation that used for continuously monitor a site and crawl updates.
On Mon, May 19, 2014 at 2:52 PM, 韩驰 <[email protected]> wrote: > Hi everyone! > > > After reading the issue: > https://issues.apache.org/jira/browse/NUTCH-1651, I have some doubts. > What is the modifiedTime and prevmodifiedTime? And is the target to > avoid fetching the same urls when fetching for a second time? > > > Thank you in advance! > -- Don't Grow Old, Grow Up... :-)
