Indeed, Klaus may very well be right, and that's why I was concerned about
being off-topic in the first place.
Anyway, I find that my own 'multipass' usage of Analog is very satisfactory,
and I understand perfectly that you would like to keep using such a great
product nevertheless. So here's what I suggest : since the long analysis
times undoubtedly result from a network bottleneck, you could try and have
you server's log preprocessed locally, generating only the info you need.
Then just run analog with this new, smaller log.
As I understand your problem, you could reduce the log size in 2 dimensions:
- decrease the number of log lines by keeping only these requests that
Analog will count as pages for you (should be similar to your PAGEINCLUDEs,
for me it is something like .htm, .html, .cgi, .php, .asp, .pdf...), since
metering your user's site usage really only involves pages and not graphics.
With this first step you'll probably find that you reduce the file size in a
1:4 or 1:5 ratio, with a direct impact on network traffic and thus analysis
times.
- one step further, you could restrict the data you have on each line: do
you need to know the params passed to the server? the bytes counts?
especially if you use a IIS server like I do, you'll find by examining the
built-in logformat called by 'LOGFORMAT MICROSOFT-whatever' that a lot of
the recorded 'info' is actually overlooked by analog ('%j', like... junk!).
(find this in the doc, LOGFORMAT section, see the examples) So to further
reduce the amount of data transiting on your lan, you might as well get rid
of it in the first place, thus creating your own custom-tailored LOGFORMAT.
All this preprocessing can be done with VBscript, since I understand you're
familiar with Visual Basic, and its support of regular expressions can prove
invaluable for this kind of job.
HTH
Bruno
----- Original Message -----
From: "Klaus Johannes Rusch" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Monday, July 14, 2003 6:20 PM
Subject: Re: [analog-help] Local Users' Requests?
> Paul Isden wrote:
>
> > I have now got this working. However...
> >
> > Each report takes 37 minutes to generate and I have 59 users! All I'm
doing
> > is creating a list of the top 50 sites each user has visited - but as
Analog
> > has to draw this year's logs from the network each time and then sift
> > through the logs to create the report - it's taking ages.
>
> If the report format doesn't matter, rewriting log entries in a
preprocessing
> step might help (for example, changing the request URL to
> http://username/http://www.example.com/)
>
> Analog is probably not the best choice for this type of analysis, a simple
> script that specificly records the information you need would run faster.
>
> --
> Klaus Johannes Rusch
> [EMAIL PROTECTED]
> http://www.atmedia.net/KlausRusch/
>
>
> +------------------------------------------------------------------------
> | TO UNSUBSCRIBE from this list:
> | http://lists.isite.net/listgate/analog-help/unsubscribe.html
> |
> | Digest version: http://lists.isite.net/listgate/analog-help-digest/
> | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
> | List archives: http://www.analog.cx/docs/mailing.html#listarchives
> +------------------------------------------------------------------------
+------------------------------------------------------------------------
| TO UNSUBSCRIBE from this list:
| http://lists.isite.net/listgate/analog-help/unsubscribe.html
|
| Digest version: http://lists.isite.net/listgate/analog-help-digest/
| Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
| List archives: http://www.analog.cx/docs/mailing.html#listarchives
+------------------------------------------------------------------------
