Hi,
I'm having trouble specifying multiple logfiles and logformats. To put
it simply, I need to generate stats for a site on a ISP that refuses to
put their logs in a standard format. Even through they use Apache, they
insist on using the 3 different access, referrer and agent logs,
however, the agent and referrer logs are not in a standard format (they
can't spell referrer either). Neither contain a date, and the referrer
log doesn't contain the page that was requested, only the referrer. So,
my agent_log looks like:
MSProxy/2.0
Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt)
Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt)
and my referer_log looks like:
http://www.mtaasuper.com.au/cgi-bin/wwwwais
http://www.mtaasuper.com.au/maintain/analog/indx.html
-
http://www.mtaasuper.com.au/
access_log is standard common format:
203.134.53.162 - - [01/Aug/2001:14:46:18 +1000] "GET
/images/ttl_employers.gif HTTP/1.1" 304 -
210.9.15.254 - mtaa [01/Aug/2001:14:47:13 +1000] "GET
/maintain/analog/favicon.ico HTTP/1.1" 200 1078
203.56.239.113 - - [01/Aug/2001:14:50:01 +1000] "GET / HTTP/1.0" 200
27359
I need to generate a report for the site each month (via a cron job). To
group the three logs by month (as agent and referrer don't contain
dates), I rotate the logs each month and save in the format
agent_log.YYYYMM.gz etc. So, to get the stats for July 2001, I need to
run analog over access_log.200107.gz agent_log.200107.gz and
referer_log.200107.gz
At first, I tried setting LOGFILE (in a .cfg file) to:
LOGFILE /home/logs/access_log.%Y%M.gz
LOGFORMAT %B
LOGFILE /home/logs/agent_log.%Y%M.gz
LOGFORMAT %f
LOGFILE /home/logs/referer_log.%Y%M.gz
but then I discovered that %'s aren't accepted in LOGFILE as they are
for CHARTDIR etc. I realised that I would have to specify the logfiles
on the command line, so I tried the command line:
/home/wwwdev/analog/analog +g/home/wwwdev/analog/config/detail.cfg
+O/home/htdocs/maintain/stats/200107_detail.html +F010701 +T010731
/home/logs/access_log.200107.gz,/home/logs/referer_log.200107.gz,/home/logs/agent_log.200107.gz
with the config file:
LOGFORMAT COMMON
LOGFORMAT %f
LOGFORMAT %B
This didn't work and it complained:
Warning D: LOGFORMAT in configuration file
/home/wwwdev/analog/config/detail.cfg with no subsequent LOGFILE
Warning F: Can't auto-detect format of logfile
/home/logs/referer_log.200107.gz: ignoring it
Broken Pipe
Warning F: Can't auto-detect format of logfile
/home/logs/agent_log.200107.gz: ignoring it
Broken Pipe
So I tried to set the LOGFORMAT on the command line also:
/home/wwwdev/analog/analog +g/home/wwwdev/analog/config/detail.cfg
+O/home/htdocs/maintain/stats/200107_detail.html +F010701 +T010731
+C"LOGFORMAT COMMON" +C"LOGFORMAT %f" +C"LOGFORMAT %B"
/home/logs/access_log.200107.gz,/home/logs/referer_log.200107.gz,/home/logs/agent_log.200107.gz
Still no luck. It mixes up the logfiles, so my referring site report
looks like the following and I don't get a browser/os report:
reqs: site
----: ----
56: http://www.mtaa.com.au/
12: http://online.socialchange.net.au/
10: mozilla/4.0 (compatible; msie 5.0; windows 98; digext;
$http://reg02.kidz.net.au/
4: http://www.jmifa.com.au/
3: http://dev.socialchange.net.au/
3: http://online.socialchange.net.au/
2: http://www.goeureka.com.au/
1: kmidc71-83.ecopost.com.au - john douglas [19/Jul/2001:22:34:42
+1000] "GET /
1: cyril.halenet.com.au - r blacker [20/Jul/2001:12:34:17 +1000] "GET
/
1: geip-a-003-pool-164.tmns.net.au - shaw brian [17/Jul/2001:20:26:40
+1000] "GET /
1: stitch.ispdr.net.au - raelene kent [14/Jul/2001:14:14:39 +1000]
"GET /
1: dc-59-6.bpb.bigpond.com - nicholas robert beminster
[15/Jul/2001:14:54:03 +1000] "GET /
1: proxy1.bne.gil.com.au - david rea [27/Jul/2001:12:02:16 +1000]
"GET /
1: proxy.octa4.net.au - deborah coleman [26/Jul/2001:11:34:05 +1000]
"GET /
1: 203.76.17.27 - o f mansour [19/Jul/2001:01:28:13 +1000] "GET /
I've tried specifying the logfiles and logformats in different orders,
but nothing seems to work. Am I doomed to failure? I can generate a
report over all of the logfiles (e.g. back to April 2000) without any
problems using:
LOGFILE /home/logs/access_log*
LOGFORMAT %B
LOGFILE /home/logs/agent_log*
LOGFORMAT %f
LOGFILE /home/logs/referer_log*
But, I also need to generate a single report for each month back to
April 2000. I could manually edit the .cfg file to generate each report
initially, but seeing as this needs to become a cron job, that's not a
long term solution.
Any help would be greatly appreciated!
Kobe Watkins
Social Change Online
+------------------------------------------------------------------------
| This is the analog-help mailing list. To unsubscribe from this
| mailing list, go to
| http://lists.isite.net/listgate/analog-help/unsubscribe.html
|
| List archives are available at
| http://www.mail-archive.com/[email protected]/
| http://lists.isite.net/listgate/analog-help/archives/
| http://www.tallylist.com/archives/index.cfm/mlist.7
+------------------------------------------------------------------------
