On Tue, 4 Apr 2000, Tristan Lawrence wrote:
>
> I think Analog is a wonderful program, but for whatever reason it uses
> an absolutely ridiculous amount of memory doing large reports like this.
> My script unquestionably takes a long time to run (hours) but is
> dramatically more memory-efficient than running Analog from logs or
> cache files. This isn't due to the fact that I throw away some of the
> data but to some inefficiency in the way Analog keeps track of things on
> this scale.
I don't think that it does use much more memory than necessary. By far the
largest amount of memory is in recording the names of every host and every
file in the logfile, even though most of them never get listed in the end.
This is maybe half the memory typically, but you can't get round it without
losing data. Then you need to record the number of times you've seen each
item, the date you last saw it, etc...
There are minor improvements you could make, but I have thought pretty hard
about this issue already, and I don't think it can be improved much.
--
Stephen Turner http://www.statslab.cam.ac.uk/~sret1/
Statistical Laboratory, 16 Mill Lane, Cambridge CB2 1SB, England
"8th March 2000. National No Smoking Day. Ash Wednesday." (On a calendar)
------------------------------------------------------------------------
This is the analog-help mailing list. To unsubscribe from this
mailing list, send mail to [EMAIL PROTECTED]
with "unsubscribe" in the main BODY OF THE MESSAGE.
List archived at http://www.mail-archive.com/[email protected]/
------------------------------------------------------------------------
