Hi,

>It might as well be a search engine spider.


Perhaps it is... By just tailing the httpd-access.log file a few times over the past 10 minutes or so, at least I already just came across:

74.6.70.45 - - [04/May/2007:13:12:34 +0200] "GET /olympus/tope/ tope_show_entry.php?event=13&pic=1 HTTP/1.0" 200 3209 "-" "Mozilla/ 5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/ slurp)"
[...]
66.249.66.176 - - [04/May/2007:13:15:46 +0200] "GET /olympus/tope/ tope_show_entry.php?pic=22&event=7 HTTP/1.1" 200 3792 "-" "Mozilla/ 5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 65.55.210.7 - - [04/May/2007:13:15:52 +0200] "GET /clomid HTTP/1.0" 404 345 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)"

So, there ya go: it looks like Yahoo!, Google AND MSNsearch are battering my server pretty much all at once (though I don't see too many traces of either of them, so perhaps they're not all too hard on the machine)...
Hmmm, perhaps time for a "robots.txt" file.

Then, doing just some more tailing on the access log shows that the actual load is spread over several sites, with about 4 of the top sites being called from a variety of browsers from all sorts of IP addresses. Another assumption then becomes that perhaps as soon as the USA "wakes up", the sites simply get a lot of real visitors too, hence causing a (legit) high load...

Still, this is precisely why I'd like to see which scripts are causing heavy load, such that I can perhaps better tune them.

Cheers,
Olafo


_______________________________________________
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Reply via email to