On Apr 9, 2000 Jeremy Wadsack writes (to Mike Vislocky)
> I didn't know you were looking at log files of less than
> 100 lines a day. I work with much larger sites and still
> have people asking the same thing -- "I want individual details
> of all the requests of the 100,000 people who visited my site
> each day."
>
> I see that in your situation the log files are a gold mine.
> And as Stephen said, for your needs, reading the logs by hand
> is probably much more effective than anything an analysis program
> could provide.
and Stephen Turner writes,
> It's the job of analog to produce summary statistics. If you want
> to see in what order an individual visitor moved round the site,
> there is no substitute to looking in the raw logfiles.
Since 1995 my two successive servers have e-mailed me as of 11:59 PM
each day, using custom code and crons, a copy of the log file for the
day and a log trace sort for the log file of the type discussed here:
alphanumeric sort of hosts, with sub-sorts of requests for each by
date time and filename. My present server also runs analog for the
same day's log, so I can download on my browser the html output file
from my virtual server first thing each morning.
I have a larger daily website access than Mike - currently 600 hosts
a day and 3,000 file requests (requesting on the average half of my
700 files). What I do, after looking at the analog General Summary,
is check the Domain and Organization Reports for unusual foreign
accesses, and the Request Report (all alphanumeric), for the unusual
numbers of file requests.
This is where the log trace sort comes in handy. I can immediately
go on log trace to the host making each unusual request and check the
files requested, and their sequence etc., as clearly set up in
columns - without having to wade through the raw log with its long,
often wrapped lines, lack of column alignment, and with confusing
interspersal of simultaneously accessing hosts, etc..
For an educational, inspirational .org website such as ours this
quantitative/qualitative analysis is indeed a "gold mine" as we see
what people are interested in, what inspires them to go to the next
file, etc. - so we can adjust our internal file-to-file hot links,
and write new text better to meet needs, etc.. Also invaluable is
the Search Query Report, as it shows what people are looking for.
(Half of our 600 daily hosts come through Internet search engines).
Any unusual search queries can be quickly checked against the Log
File (to get the URL) and then the Log Trace Report to see what file
was requested for the query (and what subsequent files the host went
on to, or not).
The advantage of having cron-activated analog output and cron
e-mailed logs and Log Trace Reports is that I have them saved and up
on my computer in two minutes each morning, and make the
above-described analyses in 15 minutes or so.
Then, I make monthly and annual analyses manually to check repeat
host accesses, trends, etc.
It seems to me, Stephen, that since we're only talking about 20 lines
of code for the log trace, and less for the log, you could find time
to add this to analog, and the whole thing could be controlled using
analog.cfg. I've got a pretty good set-up already, but it would even
better if was all done through analog (time saved saving files,
switching back and forth, etc.)
When I wrote to the list last summer requesting the same (in detail,
with examples) it elicited minimal interest, but with others making
the same request now, maybe you'll re-consider?
And if one of one of Jeremy's customers asking for details of the
100,000 daily hosts is a big bucks customer, he could send them an
excerpt copy of the Log Trace file!
John Stokes
+------------------------------------------------------------------------
| This is the analog-help mailing list. To unsubscribe from this
| mailing list, go to
| http://lists.isite.net/listgate/analog-help/unsubscribe.html
|
| List archives are available at
| http://www.mail-archive.com/[email protected]/
| http://lists.isite.net/listgate/analog-help/archives/
| http://www.tallylist.com/archives/index.cfm/mlist.7
+------------------------------------------------------------------------
