I have a system that does something similar to this... Web logs are fed via a pipe to a batcher program. The batcher, among other things, multicasts all of the log data out on a multicast port. I have a script that listens to that multicast traffic (we have several web servers) and counts things like pageviews, hits, etc. It updates an rrd every five seconds.
I'll attach the metering script, but you'll have to come up with your own batching script (it's not releasable at the moment). The script expects to receive log data that looks like this: site access www2 165.230.46.168 - - [02/May/2002:14:17:03 -0600] "GET /js/city-list.fhtml?loc=NJ HTTP/1.0" 200 4245 "http://www.flipdog.com/js/loc.html?_requestid=731741"; "Mozilla/4.77 [en] (Windows NT 5.0; U)" "www.flipdog.com" site access www2 206.53.0.186 - - [02/May/2002:14:17:03 -0600] "GET /images/nav/flipdog_support.gif HTTP/1.1" 200 343 "http://www.flipdog.com/"; "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)" "www.flipdog.com" site access www3 152.163.207.46 - - [02/May/2002:14:17:03 -0600] "GET /js/footer.fhtml?step=1 HTTP/1.0" 200 5916 "http://www.flipdog.com/js/loc.html;$sessionid$RKQUSZXLF20D3QFIDAVSFEQ?_requestid=731699"; "Mozilla/4.0 (compatible; MSIE 5.5; CS 2000; Windows 98)" "www.flipdog.com" where the first word on the line indicates that this is site traffic, the second word indicates that it is access log information (as opposed to error or ssl logs), and the third word tells which web server kicked out this log entry. After those three keywords, it's basically an apache extended log format log line. The script also interactively dumps it's counts for each web server every five seconds. I run it in a screen session that I can reattach to if I need to look at how the site is performing at the moment. -jan- -- Jan L. Peterson <[EMAIL PROTECTED]> -- Attached file removed by Listar and put at URL below -- -- Type: text/plain -- Desc: meter_rrd -- Size: 2k (2638 bytes) -- URL : http://www.ee.ethz.ch/~slist/pantomime/meter_rrd -- Unsubscribe mailto:[EMAIL PROTECTED] Help mailto:[EMAIL PROTECTED] Archive http://www.ee.ethz.ch/~slist/rrd-users WebAdmin http://www.ee.ethz.ch/~slist/lsg2.cgi
