Author: Alexander Email: Barkov Message: 3.1.x caches robots.txt information in SQL table robots. Remove www.wwf.org.uk entries from this table.
> I am trying to index this address but it contains a robot.txt which prevents me from >indexing this directory > > Server http://www.wwf.org.uk/news/ > > when i add "Robots no" to indexer i get the message > > indexer[1744]: [1] robots.txt support is disallowed for 'www.wwf.org.uk' > > CAn anyone help me index this directory > > Reply: <http://www.mnogosearch.org/board/message.php?id=3260> ___________________________________________ If you want to unsubscribe send "unsubscribe general" to [EMAIL PROTECTED]
