- - - - - - - - - - - - - - - - - - - - - - - - - - - -
Name: alex
Subject: bugs in indexer (crawling)

(using the latest snapshot)

I have some:

Server world url1
Server world url1

Then: Realm regexp smth

why is the indexer going out of the regexp and hits robots.txt or fetches pages 
which are clearly outside the Realm?

10x
- - - - - - - - - - - - - - - - - - - - - - - - - - - -

Read the full topic here:
http://www.dataparksearch.org./cgi-bin/simpleforum.cgi?fid=03;post=

Ответить