Author: Alexander Barkov
Email: [EMAIL PROTECTED]
Message:
Hi everybody,
How can I exclude a directory from de indexing process?
Using allow/disallow commands, or using robots.txt file
if you are indexing your own site.
Reply: http://search.mnogo.ru/board/message.php?id=1541
Author: Antonio Jose Moran
Email: [EMAIL PROTECTED]
Message:
Thanks,
I have just used allow/disallow, but not robots.txt (where can I find information
about "robots.txt" I have udmsearch 3.0.23)
The problem is that I want to index a subdirectory that belongs to a directory
that is
Author: Alexander Barkov
Email: [EMAIL PROTECTED]
Message:
Robots exclusion standard can be found here:
http://info.webcrawler.com/mak/projects/robots/robots.html
Check indexer.conf-dist how to disallow subdirectories.
Thanks,
I have just used allow/disallow, but not robots.txt (where
