Author: Alexander Barkov
Email: [EMAIL PROTECTED]
Message:
> How do folks handle situations in your string and regular expressions where a web 
>server can handle multiple backslashes in a url?
> 
> example:
> 
> http://www.uvm.edu//foo
> http://www.uvm.edu///foo
> http://www.uvm.edu////foo
> 
> all equal
> http://www.uvm.edu/foo
> 
> which is already handled by a disallow command or the robots file.
> 


Try something like this:

Disallow http://*//*


Reply: <http://www.mnogosearch.org/board/message.php?id=4295>

___________________________________________
If you want to unsubscribe send "unsubscribe general"
to [EMAIL PROTECTED]

Reply via email to