Raghavendra Prabhu wrote:
Thanks Andrez
One more thing which i wanted to know was fetching the last modified date is
done as a part of content parsing function.
Cant we split the whole thing into two funtions
1)The first function will getmodified date
2) The second function will get content
Initiall the first function will be called to get the modified date and
compared against crawldb modified date and only if it is different will the
second function (which actually fetches the content) will be called
Will this be the motive of adpative interval patch?
Yes, this is included in that patch.
--
Best regards,
Andrzej Bialecki <><
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Nutch-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-general
