I found the problem... it was not Analog... it was ReportMagic, when it was building a report for a site which log file was 930 MB. My solution was to cut down the number of results displayed. It was configured for ALL results, so, this site had millions of HOSTS addresses... That made ReportMagic create a hosts.html file over 8 MB.
Well, I fixed this by setting the FLOOR to -250r. Now, it too 3 HS to process 400 log files, from various sizes (from 1 MB to 580 MB).
I uses RDNSLOGS, a helper app, which is GREAT.
Thanks!
From: "David McCormack" <[EMAIL PROTECTED]> Reply-To: [EMAIL PROTECTED] To: <[EMAIL PROTECTED]> Subject: RE: [analog-help] Running Analog every day suggestion... Date: Wed, 6 Aug 2003 19:28:06 +0100
I run Analog over ~ 1600 log files (from an Apache server) most are about 200K and it takes about 10 minutes to process them all, and then run the output with Report Magic. The files are dragged across a network (100-base-TX) which slows things down slightly. I don't have to do the DNS lookup though, as that�s taken care of by my provider.
When I have used the Analog's inbuilt DNS lookups it was slow. Are you using
a DNSFILE and DNS WRITE commands? The first time you use them it doesn't
help much but the second time you do notice the difference.
You could try one of the helper applications (http://www.analog.cx/helpers/)
they might help you as well.
HTH
David McCormack
-----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Agustin Chernitsky Sent: 05 August 2003 14:49 To: [EMAIL PROTECTED] Subject: [analog-help] Running Analog every day suggestion...
Hi guys,
I run Analog every day for about 100 log files (all IIS). The performance is
great, but I must doing something wrong, because it takes 6 HS to finish all
the log files.
I rotate the log files every month, and delete logs older that 4 months. So
analog is configured to analize 4 months per site, including the rotated log
files.
Is this approach correct? I believe this is why it�s taking so long, because
analog must process all log files (400 log files) and run DNS queries for
all of them.
Can anyone suggest a better way to achieve this?
The objective is show reports by quarters (4 months).
Thanks!!
Agustin.
_________________________________________________________________ The new MSN 8: advanced junk mail protection and 2 months FREE* http://join.msn.com/?page=features/junkmail
+----------------------------------------------------------------------- +- | TO UNSUBSCRIBE from this list: | http://lists.isite.net/listgate/analog-help/unsubscribe.html | | Digest version: http://lists.isite.net/listgate/analog-help-digest/ | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general | List archives: http://www.analog.cx/docs/mailing.html#listarchives +----------------------------------------------------------------------- +-
+------------------------------------------------------------------------ | TO UNSUBSCRIBE from this list: | http://lists.isite.net/listgate/analog-help/unsubscribe.html | | Digest version: http://lists.isite.net/listgate/analog-help-digest/ | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general | List archives: http://www.analog.cx/docs/mailing.html#listarchives +------------------------------------------------------------------------
_________________________________________________________________
Tired of spam? Get advanced junk mail protection with MSN 8. http://join.msn.com/?page=features/junkmail
+------------------------------------------------------------------------ | TO UNSUBSCRIBE from this list: | http://lists.isite.net/listgate/analog-help/unsubscribe.html | | Digest version: http://lists.isite.net/listgate/analog-help-digest/ | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general | List archives: http://www.analog.cx/docs/mailing.html#listarchives +------------------------------------------------------------------------
