Hi Dave
> I am a marketing consultant working with a range of clients who > use log file analysis tools such as WebTrends and RedSheriff. > Both of these tools (and others) use (or at least have an option > to use) cookies in order to track unique visitors to web sites. > My understanding is that search engine spiders cannot accept > cookies... Most search engine spiders won't accept cookies for various reasons, it's true. > ... and consequently will not trawl a site that serves > cookies to them. That's not true. It's down to the site (i.e. some component of the site hosting/serving architecture) more than the search engine spider. If the site requires the client to accept cookies, and the client (in this case, the search engine spider) can't or won't accept cookies, clearly the client won't be allowed to get round the site. But Webtrends (and, I guess, RedSheriff, although I've no direct experience of that) does not *require* cookies, just uses them if available. WebTrends Live even sets its cookies with JavaScript, so search engine spiders (which rarely interpret JavaScript) won't even see the Set Cookie request. Search engine spiders have no trouble crawling sites simply because those sites use WebTrends, although they may have trouble crawling the same sites for lots of other reasons. > I understand that cloaking is a POSSIBLE > solution - however this in itself raises a few questions for me: > > 1) If cloaking is an acceptable method to get round the problem > of sending away a search engine spider (and consequently not > indexing pages within a site), how should this technology be > implemented and what pitfalls need to be considered. Cloaking is not an acceptable method of getting round the problem. In fact, you probably don't have a problem anyway. > 2) Most search engines state clearly that they do ban sites > altogether from their indexes if they discover cloaking has been > used? Are there any other methods that are preferable? Cloaking is definitely frowned upon by search engines and I would never advocate its use. I've written a White Paper on Cloaking and Spam at http://www.ebrandmanagement.com/whitepapers/spam1.htm if you're interested in the issues that search engines have with this. The short of it is that there is always a better answer that (as well as being more ethically sound) usually provides greater long-term marketing success than cloaking offers. > I have to say that although I have a technical background I am > not a programmer or software developer - but I have been learning > a lot simply by subscribing to this list - keep up the great > work. However, I am looking for a reasonably technical answer (or > where I might go to find one please) so that I can pass on > specific suggestions to my client, WebTrends and RedSheriff. If you need more, send me an e-mail (on or off-list) - like you, I'm based in the UK so this may be quicker handled on the phone. Regards Alan Perkins Chief Technology Officer, e-Brand Management Limited http://www.searchmechanics.com/ - Expert search engine advice, training and software tools -- This message was sent by the Internet robots and spiders discussion list ([EMAIL PROTECTED]). For list server commands, send "help" in the body of a message to "[EMAIL PROTECTED]".
