On Wed, 26 Jan 2005 09:44:57 +1300 (NZDT) John Carter <[EMAIL PROTECTED]> wrote:
> On Wed, 26 Jan 2005, Nick Rout wrote: > > >>> Mozilla/5.0 (compatible; Googlebot/2.1+http://www.google.com/bot.html) > >> > >> Ooo. I _like_ that! So all those snooty sites that hide their content > >> behind "You need IE to view this site" are not getting googled. Yet > >> another reason to love Google! > > > > I'm not so sure that such silliness prevents spidering > > Oh but it does, the class of site I'm talking about checks the UserAgent > string and refuses to display it content unless it is being browsed by IE. > > So if Google is reporting Mozilla as its useragent, then this class of > broken site will simply give Google it's content/internal link free > "install IE" page. > > There is no way for Google to spider past it! > > Instant web site suicide. > > Think of it as evolution in action. Nice thought, but I suspect google are cleverer than that! if i were to design their robot i would make it change its user agent string on receipt of such a response. > > > > John Carter Phone : (64)(3) 358 6639 > Tait Electronics Fax : (64)(3) 359 4632 > PO Box 1645 Christchurch Email : [EMAIL PROTECTED] > New Zealand > > "The notes I handle no better than many pianists. But the pauses > between the notes - > ah, that is where the art resides!' - Artur Schnabel -- Nick Rout <[EMAIL PROTECTED]>
