"Claudiu Costin" <[EMAIL PROTECTED]> wrote:
> Again, you don't read emails with attention. _Please_ take a 300M log
> file on your HDD, make your nice script and send us your CPU, mem
> loading/benchmarks and your computer specs (cpu mhz, mem, O.S. etc).
> Are you think I'm stuppid? Make this then
You're obviously confused - you're the one who wants a change made, the
onus is on you to do the grunt work to justify your position.
But, just to humour you, I grabbed the logs off one of our servers -
433MB, April 1st to April 7th. I picked an hour at random, 3-4PM on the
5th, and extracted all lines for that hour to a separate file. It took
approximately 2 minutes and 25 seconds, and generated a 4MB log file:
FINDSTR /C:"2001-04-05 15:" ex01040?.log > hour.log
It took about 2 seconds to get a host count on this with:
perl -lane "print $F[2]" hour.log | sort | uniq | find /c "."
(I did this in two passes, with a temporary file, but I could have done it
in a single pass, just piping the data between processes if I'd wanted
to).
Just using analog with
LOGFILE ex0104*.log
HOST ON
FROM 010405:1500
TO 010405:1600
took approximately 1 minute and 45 seconds to get a similar result, most
of which was because I forced Analog to read a whole weeks logs. If I'd
told Analog to just look at the relevant logfile, it probably would only
have taken 20 seconds, and because the 70M daily file would probably be
cached in memory, I could probably have run 24 iterations against the
daily file in 4 or 5 minutes, and gotten hourly host counts for the whole
week in less than an hour.
I did this on a P6/600Mhz with 128Mb of RAM running NT (I didn't bother
shutting down any of the 15 or so windows I had open before I started
this). Hardly a powerhouse machine at this stage of the game.
I did all this with command line tools, and most of the time was spent
just reading through the 433MB for data. If I really wanted a host count
for each hour, it would probably be a fairly trivial task to write a
script that would do a host count at each hour transition, getting all 168
host counts in a single pass. It would run in less time than it's taken to
read this thread (and it probably wouldn't take much longer for a
competent perl programmer to program it - I could probably do it myself in
a couple of hours if I wanted it, and I even had to look up the little bit
of perl that I did use).
Aengus
+------------------------------------------------------------------------
| This is the analog-help mailing list. To unsubscribe from this
| mailing list, go to
| http://lists.isite.net/listgate/analog-help/unsubscribe.html
|
| List archives are available at
| http://www.mail-archive.com/[email protected]/
| http://lists.isite.net/listgate/analog-help/archives/
| http://www.tallylist.com/archives/index.cfm/mlist.7
+------------------------------------------------------------------------
