I think better way Matthias idea: dedup segments.
In the older then 30
days segments you can found not changed pages, thats are not exists in
the new segments.
No, pages become refetched after 30 days. So they will be in the new
segments. You could remove segments after 30 + x days.
Matthias
--
http://www.eventax.com - eventax GmbH
http://www.umkreisfinder.de - Die Suchmaschine f�r Lokales und Events
-------------------------------------------------------
This SF.net email is sponsored by Demarc:
A global provider of Threat Management Solutions.
Download our HomeAdmin security software for free today!
http://www.demarc.com/Info/Sentarus/hamr30
_______________________________________________
Nutch-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-general
