-----Original Message-----
From: li...@webstandardsgroup.org [mailto:li...@webstandardsgroup.org]
On Behalf Of Philippe Wittenbergh
Sent: 01 November 2009 23:05
To: wsg@webstandardsgroup.org
Subject: Re: [WSG] [OT] Google search/index/webmaster help

>  Because that file is being served as 'text/html' instead 
>  of 'text/xml' as it should. 
>  That is server misconfiguration. 
I agree that this is technically incorrect, but hardly unusual.

>  I'm not surprised Googlebot doesn't pick it up.
I would be absolutely flabbergasted if Google ignored it purely because
of that.
Google has a strong history of being pragmatic; they _want_ to use this
file; why would you expect them to ignore a file with the right name,
the right kind of content, in the right place?

As an aside, how many robots.txt do you think get served up as text/html


Mike Brockington
Web Development Specialist


This message does not reflect the opinions of any entity other than the
author alone.

List Guidelines: http://webstandardsgroup.org/mail/guidelines.cfm
Unsubscribe: http://webstandardsgroup.org/join/unsubscribe.cfm
Help: memberh...@webstandardsgroup.org

Reply via email to