This is what the Analog cache files are for (see http://analog.cx/docs/cache.html). You can create a cache file for each day's results and then write reports by reading in the cache files. This will certainly improve performance. It may or may not help memory problems.
If you are running out of memory, you can try using the *LOWMEM commands (see http://analog.cx/docs/lowmem.html). Generally the Host and Referrer data are the most memory consumptive. You can also add more memory to the box running Analog (memory is pretty cheap these days). You can't just run reports and use those to produce other reports because they don't contain complete information. Because the reports only show the top items, they will not properly count items not included in particular reports. Once you have exhausted all these options there are a couple additional things you can try. We configured one large client's processing with very careful use of *LOWMEM 3 commands (which remove data from the reports). You can run a successive series of reports on each data type and then recombine them. This will take a long time (about 5 - 6 times as long as the original processing) but uses less memory. It's imperative, however, that you do conversion testing on known data to guarantee that you are gathering and calculating everything correctly in the process. A more effective approach is to look at your data and make some decisions about what you *really* need to report on and what you don't. For example, how useful is and annual report? Do you need to track all the graphics files in your reports (bandwidth tracking is better handled through network monitoring apps anyway)? Can you get more effective reports by splitting the data into domains based on target audiences or user groups? Finding effective ways to implement *INCLUDE/*EXCLUDE and *ALIAS command to reduce the data volume that Analog processes can produce long-term (assuming a growth trend) approaches to data management. -- Jeremy Wadsack Wadsack-Allen Digital Group Monsur Hossain <[EMAIL PROTECTED]> (Tuesday, September 09, 2003 8:27 AM): > Not sure about doing this through Analog, but I wrote a helper app to kick > off analog. Once analog is done running, the helper app uses regex to parse > the output html and grab the relevant data I wish to track over longer > periods of time (successful reqs, redirected reqs, failed reqs and data > transferred). > Monsur > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Cal Lidderdale > Sent: Monday, September 08, 2003 4:59 PM > To: [EMAIL PROTECTED] > Subject: [analog-help] File size problem > Sounds too good to be true - its not what one would think - we're using > Apache as a data server [ ie: vox files et-al ] and the requests run in the > 1/2 Mil / day range with todays access_log at 60 Mb and yesterdays at > 60 Mb and .... Like da guy sayz "Billion here, billion there, pretty > soon we'er talk'n real money." > So my question is: Is there a way to run "a" day summary that would be > equal to a one day web page and store it? FF or DB. Then when I run "analog .... >> htdocs/Stats/THE-WHOLE-YEAR.html" it would combine each days > "summary" into a report. This way it a) wouldn't take 3 hrs and b) won't > say "out of memory" :'( > Tks > Chuck +------------------------------------------------------------------------ | TO UNSUBSCRIBE from this list: | http://lists.isite.net/listgate/analog-help/unsubscribe.html | | Digest version: http://lists.isite.net/listgate/analog-help-digest/ | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general | List archives: http://www.analog.cx/docs/mailing.html#listarchives +------------------------------------------------------------------------
