Good point, and thanks for your comments-not to mention the lucid
example!
I'm not sure that would be a problem, though. As I recall (I wrote this
a while ago) the script collects all of the stats for a particular day;
it doesn't limit them at all. The filtering takes place when it outputs.
Anyway, I should definitely have said that this is a
use-at-your-own-risk type of thing. At the same time, I would love to
hear any thoughts anyonbe has on how it could be improved.
Tristan
Jeremy Wadsack wrote:
>
> Tristan,
>
> You have to be careful with this script: The problem is that each month only
> contains the collected stats of each day. This means the data at the bottom of
> each time report may not be accurate (in fact all the numbers could be
> inaccurate). When you produce the reports, Analog cuts off the bottom of the
> report at the floor. Some items may never make the reports that should, others
> could be miscounted. Here are two examples to highlight it:
>
> Say that the bottom of the request report, every day of the month is at about 1.5
> million requests. A particular file, sales.html, has about 1.4 million requests
> per day. It never makes the cut off. Now, assume there's a lot of different
> activity on the site. Many of the pages get 2 million requests some days and 200
> request others. At the end of the period, the top 200 requests in your stats could
> show some of these latter items, which may have only totaled 3 million requests,
> yet sales.html, with a total of 43.4 million requests is never listed (though it
> should be) 'cause it was never in any daily report.
>
> Next, assume you've been running a service at program/login.cgi. At the end of the
> month, your ad campaign finally runs and tells people about that file. While the
> rest of the month this was only getting 800 thousand requests a day, in the first
> day after the campaign runs, this reaches 10 million requests. The next two days
> (before the end of the month) this logs 8 million requests a day. The last three
> days it is listed in the daily reports and totals in your report at 26 million, so
> it doesn't make the month-end report (which cuts off at 30 million). But really,
> it's been getting hits all along. The actual amount should be 48.4 million. That's
> a big difference to marketing!
>
> So, I'm not saying don't use the script. Just be certain you understand the
> limitations and implications. This may be the only solution for huge logfiles like
> you both have. Of course might be able to use something like Marco's script (or a
> set of equivalent Analog configs) in the separate, concurrent thread "Unique user
> counts per request" to report on parts of the logs that you need.
>
> HTH,
>
> Jeremy Wadsack
> Wadsack-Allen Digital Group
>
> Tristan Lawrence wrote:
>
> > I have attached a perl script I used to get around a similar problem
> > (although I only had 1.5 GB of memory not 14 Gb!!!)
> >
> > It runs each day seperately and combines the output.
> >
> > Tristan
> >
> > Fiorela Gonzalez wrote:
> > >
> > > I am running analog on a machine that has 14G of memory, disk space a couple
> > > of tarabytes. I am trying to run a monthly report (just for march) and i
> > > keep running out of memory. I get this message from the my error log. And
> > > Yes I have set hostlowmem to 3 and filelowmem to 3. I also have cache files.
> > > A daily takes about 30 min to 45 min to run. A weekly takes about 2 hours to
> > > run (this is with hostlowme 2 and filelowmem 2. So monthly usually dies at
> > > around 3 hours or so. Any suggestions?
> > >
> > > Fiorella Gonzalez
> > > Xdrive.com
> > > Intranet/Stat Manager
> >
>
> ------------------------------------------------------------------------
> This is the analog-help mailing list. To unsubscribe from this
> mailing list, send mail to [EMAIL PROTECTED]
> with "unsubscribe" in the main BODY OF THE MESSAGE.
> List archived at http://www.mail-archive.com/[email protected]/
> ------------------------------------------------------------------------
------------------------------------------------------------------------
This is the analog-help mailing list. To unsubscribe from this
mailing list, send mail to [EMAIL PROTECTED]
with "unsubscribe" in the main BODY OF THE MESSAGE.
List archived at http://www.mail-archive.com/[email protected]/
------------------------------------------------------------------------
