Is it possible to configure htdig so that it will follow links off the main
site, but limit how many pages off the start page it will go?
for example:
say i have www.somesite.com/page.html and page.html has links on it to 10
other sites. I would want htdig to crawl those 10 other sites, but only
crawl them to a depth of say 3 links off my start page.
_______________________________________________
htdig-general mailing list <[EMAIL PROTECTED]>
To unsubscribe, send a message to <[EMAIL PROTECTED]> with a
subject of unsubscribe
FAQ: http://htdig.sourceforge.net/FAQ.html
