Piotr Kosiorowski wrote:
I started to investigate -reftechonly flag because of some questions on
nutch-user mailing list. I was sure it works as described in one of the
emails on the list:
"-refetchonly generates you an segment(FetchList) that only contains the
urls that need to be refetched based on your refetch interval.
Right, new discovered links are not in the fetchlist that will be
generated by using this option."
The original rationale for the "-refetchonly" option was to permit
indexing of all of the urls known the the database, with anchor text,
but without fetching them. Thus one can, e.g., provide an index of 10M
urls while only actually fetching 1M urls. I have never actually used
this feature myseufl. I don't know whether other folks have ever used
it sucessfully, nor whether such a feature is in fact desired.
Doug
-------------------------------------------------------
This SF.Net email is sponsored by: NEC IT Guy Games. How far can you shotput
a projector? How fast can you ride your desk chair down the office luge track?
If you want to score the big prize, get to know the little guy.
Play to win an NEC 61" plasma display: http://www.necitguy.com/?r=20
_______________________________________________
Nutch-developers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-developers
