Author: Tim Hewitt
Email: [EMAIL PROTECTED]
Message:
Hi,
Have you given any more thought to adherance to the robots exclusion standard?
The document at:
http://www.robotstxt.org/wc/norobots-rfc.html
Clearly states that:
Disallow /tmp
would disallow access to tmp.html and should be treated as a simple prefix match. I'd
like to disallow links like /calendar.php?s=123123, but I'm not able to do so.
Thanks,
-Tim
Reply: <http://search.mnogo.ru/board/message.php?id=2072>
___________________________________________
If you want to unsubscribe send "unsubscribe general"
to [EMAIL PROTECTED]
