Webboard: Limit crawl to only 1 hop
Author: gluke Email: Message: I try to limit crawl to only 1 hop from the initial site. I want to completely crawl : http://mysite.com/ So , i dont understand - you want to completely crawl this site or just one hop from it ? and i want to completely crawl the external site that it refer to. If it refer to only one site, it should be easire do add this site to Server command too. I don't want to crawl other sites referer by external sites. You can use Disallow/Allow directivs to allow only small set of sites and disallow anything else. Reply: http://www.mnogosearch.org/board/message.php?id=5631 ___ If you want to unsubscribe send unsubscribe general to [EMAIL PROTECTED]
Webboard: Limit crawl to only 1 hop
Author: phil Email: Message: I try to limit crawl to only 1 hop from the initial site. I want to completely crawl : http://mysite.com/ and i want to completely crawl the external site that it refer to. I don't want to crawl other sites referer by external sites. I Try this : Follow world http://mysite.com/ Follow site Realm * Pb : the crawler is go back to index the entire world ! I try this : Follow world http://mysite.com/ Follow no Realm * But this crawl only one page from the external site. is there one solution ? Thanks phil Reply: http://www.mnogosearch.org/board/message.php?id=5626 ___ If you want to unsubscribe send unsubscribe general to [EMAIL PROTECTED]
