Hi Folks,
I have a start_url list of about 80 sites, and the same number for
limit_urls_to. I have set max_hop_count to zero. Well, to make a long story
short, htdig had indexed over 8000 docs before I killed it this morning.
With a smaller url list, and setting max_hop_count to zero, only the
initial pages are indexed, as expected. I've already had a tremendous
wrestling match with this task, so until I narrow down the list to find out
what number/site is causing this can anybody tell me if there are known
troubles in this area of code ?
Many thanks,
Denis
[EMAIL PROTECTED]
------------------------------------
To unsubscribe from the htdig mailing list, send a message to
[EMAIL PROTECTED] containing the single word "unsubscribe" in
the SUBJECT of the message.
