Mary's Gardens wrote:
> Since 1995 my two successive servers have e-mailed me as of 11:59 PM
> each day, using custom code and crons, a copy of the log file for the
> day and a log trace sort for the log file of the type discussed here:
> alphanumeric sort of hosts, with sub-sorts of requests for each by date
> time and filename. My present server also runs analog for the same
> day's log, so I can download on my browser the html output file from my
> virtual server first thing each morning.
>
> I have a larger daily website access than Mike - currently 600 hosts a
> day and 3,000 file requests (requesting on the average half of my 700
> files). What I do, after looking at the analog General Summary, is
> check the Domain and Organization Reports for unusual foreign accesses,
> and the Request Report (all alphanumeric), for the unusual numbers of
> file requests.
>
> This is where the log trace sort comes in handy. I can immediately go
> on log trace to the host making each unusual request and check the files
> requested, and their sequence etc., as clearly set up in columns -
> without having to wade through the raw log with its long, often wrapped
> lines, lack of column alignment, and with confusing interspersal of
> simultaneously accessing hosts, etc..
> ...
> It seems to me, Stephen, that since we're only talking about 20 lines of
> code for the log trace, and less for the log, you could find time to add
> this to analog, and the whole thing could be controlled using
> analog.cfg. I've got a pretty good set-up already, but it would even
> better if was all done through analog (time saved saving files,
> switching back and forth, etc.)
>
> When I wrote to the list last summer requesting the same (in detail,
> with examples) it elicited minimal interest, but with others making the
> same request now, maybe you'll re-consider?
What you are asking for is a "multi-variate" (or cross-reference)
report. Something Analog explicitly does not do (this is in the FAQ,
which I'm sure you have read).
However, if I understand your log trace correctly, this kind of thing
could be done easily by running a report with HOSTINCLUDE for the
specific host that you want to look at (as Stephen suggested, already).
If you setup the form interface to Analog, you wouldn't even have to
download the log trace every night. You could just pull up the form
interface and plug in the interesting hosts and produce reports on-the-fly.
> And if one of one of Jeremy's customers asking for details of the
> 100,000 daily hosts is a big bucks customer, he could send them an
> excerpt copy of the Log Trace file!
Well, running this on large logfiles may not make sense. Unique hosts
are a huge set, so generating a trace for each host becomes a waste of
processing power when you are only looking at a couple. (And although I
don't know your software, I know how long it takes Analog to do
summaries on these data, so there might not be time to build a log trace
for all hosts.)
Again, we go back to running Analog with HOSTINCLUDE to look at just the
information that is useful. I find that I use Analog a lot as a
command-line tool (and ASCII output) to find specific details in log
files because the filtering is very fast and the reports have almost all
the details one could need (when filtered properly).
Obviously, there are several approaches to getting this information. For
smaller files (where Analog can produce reponses in a reasonable time)
using the form interface can be quite instructive as it allows you to
try out different filters and produce different views of the data in the
log files.
I also now realize the efficacy of reading logfiles themselves for
smaller sets. I had previously discounted this because on larger sites,
with many unique hosts, subsequent hits from a given host are not near
to each other and the likely hood of a single visitor coming from a
single IP address is much lower (proxies, caches, etc.). So now I have a
new tool in my bag of tricks.
Thanks,
--
Jeremy Wadsack
Wadsack-Allen Digital Group
+------------------------------------------------------------------------
| This is the analog-help mailing list. To unsubscribe from this
| mailing list, go to
| http://lists.isite.net/listgate/analog-help/unsubscribe.html
|
| List archives are available at
| http://www.mail-archive.com/[email protected]/
| http://lists.isite.net/listgate/analog-help/archives/
| http://www.tallylist.com/archives/index.cfm/mlist.7
+------------------------------------------------------------------------
