- - - - - - - - - - - - - - - - - - - - - - - - - - - - Name: Sharon Subject: CrawlDelay
My indexer.conf contains two sites: Realm Allow http://www.A.com/forum* Realm Allow http://www.B.com/* URL http://www.A.com/forums.php URL http://www.B.com/ I set at the end of indexer.conf CrawlDelay 10 I was expecting a page download for every 10 seconds. But when i run the indexer, it crawls without any delay Runing: "indexer -p 10000" fixed it. But still, since as i crawl for two different sites i was expecting interleaving between the requests, i.e. every 10 sec download one page from site A _and_ one page fom site B ? Isnt it supposed to be like that ? What am i doing worng ? TIA ! - - - - - - - - - - - - - - - - - - - - - - - - - - - - Read the full topic here: http://www.dataparksearch.org/cgi-bin/simpleforum.cgi?fid=02;post=
