tags 377142 + wontfix
thanks

Hello,

I see at least two good reasons to prevent crawlers from seeing
alternative page formats :
1. A search engine may decide to present the [raw|print] version as 
   the best result, rather than the html version (Google does this
   sometimes !).
2. In most circumstances, the downloaded alternative format will not
   be desired by the visitor.
3. This would increase the workload on the server.

Suggested workaround : Make "wget" to present an alternative User-agent
string, so it isn't recognized as a crawler.




-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Reply via email to