On Wed, Jun 17, 2009 at 10:45 AM, Chad Etzel <[email protected]> wrote:
> > Hi All, > > Has anyone created a list or kept track of all the > services/bots/crawlers that ping a site as soon as a link is posted to > twitter? They affect user-facing pageview stats since the bots don't > actually "view" the pages. Catching this at an IP or user-agent level > would be handy... I have one of those, but I can't quite imagine how anybody would compile a list... except, now that I think about it, I should be able to figure it out from my own site's logs. I'll take a shot at that and post here when I get something. I've been using script-based analytics and I'm assuming that most or all such bots aren't going to show up there. A quick look at my logs shows a bunch of non-browser user agents. Nick
