Raghavendra Prabhu wrote:
For I run an indexing process which take 5 hours and two urls are crawled
(one at the start and one at end)
Will these url have different dates or it is the time in which it is updated
into db?
If it is the second case ,all the urls will have the same initial time i
guess.
Each entry in the segment has a timestamp. The timestamp is recorded
when Fetcher finishes fetching a given url. So, in a single segment
these timestamps will be different - the first one will show you when
you started crawling that segment, and the last one when you finished.
--
Best regards,
Andrzej Bialecki <><
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Nutch-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-general
