Re: [Dspace-tech] Regenerate monthly reports

2011-10-21 Thread Mark H. Wood
On Wed, Oct 19, 2011 at 09:18:18AM -0400, Mark H. Wood wrote: It would be nice if the stat. converters and loaders were able to run as filters, so you could do something like: gunzip foo.log | dspace stats-log-converter | dspace stats-log-importer but no: ClassicDSpaceLogConverter

Re: [Dspace-tech] Regenerate monthly reports

2011-10-19 Thread Alan Orth
Thanks for the replies... I just gave this a go. Wow. I had been zipping the historical log files in a cron job, so I had to unzip them to do the log analysis. I had 25 gigs free, but ran out of disk space unzipping them! I didn't anticipate them taking that much space... can the log

Re: [Dspace-tech] Regenerate monthly reports

2011-10-19 Thread Sean Carte
On 19 October 2011 14:55, Alan Orth alan.o...@gmail.com wrote: Wow.  I had been zipping the historical log files in a cron job, so I had to unzip them to do the log analysis.  I had 25 gigs free, but ran out of disk space unzipping them! Or you could process them individually with something

Re: [Dspace-tech] Regenerate monthly reports

2011-10-19 Thread Mark H. Wood
On Wed, Oct 19, 2011 at 03:55:27PM +0300, Alan Orth wrote: Thanks for the replies... I just gave this a go. Wow. I had been zipping the historical log files in a cron job, so I had to unzip them to do the log analysis. I had 25 gigs free, but ran out of disk space unzipping them! It

Re: [Dspace-tech] Regenerate monthly reports

2011-10-19 Thread Brian Freels-Stendel
Morning (at least, here), Well, it gets tricky. You'd have to either take a copy of the db along or allow access to the production db. If, that is, you want titles to show up in the list of items accessed instead of handles. After that, I'm not sure exactly how you'd get it to pick up on

Re: [Dspace-tech] Regenerate monthly reports

2011-10-18 Thread Alan Orth
Brian, Thanks for the vote of confidence. I'll give this a try tonight when our servers aren't as busy. Another question, there's nothing that gets modified in the database when this happens, so I shouldn't need to restart Tomcat, right? Thanks, Alan On Mon, Oct 17, 2011 at 5:34 PM, Brian

Re: [Dspace-tech] Regenerate monthly reports

2011-10-18 Thread Brian Freels-Stendel
Hi Alan, That's right, Tomcat won't need to be restarted. The stats scripts do access the database to get titles of items, but it doesn't alter anything, so there should be no side effects. I'd say good luck, but you won't need it, big smile. B-- On 10/18/2011 at 1:11 AM, in message

Re: [Dspace-tech] Regenerate monthly reports

2011-10-18 Thread Stuart Lewis
Hi Alan, Another question, there's nothing that gets modified in the database when this happens, so I shouldn't need to restart Tomcat, right? Yes - that is correct. Everything happens on disk (not in the DB): .log files -- .dat files -- .html reports The .html reports are then loaded

Re: [Dspace-tech] Regenerate monthly reports

2011-10-17 Thread Alan Orth
Hi, Never heard back on this, so I'm re-sending: We had some bad metadata and didn't realize for a few weeks that our stats scripts were choking. Now we have a gap in our monthly stats (08/2011, 10/2011... but no 09/2011!) Is clearing the stats and rebuilding from scratch feasible? All the

Re: [Dspace-tech] Regenerate monthly reports

2011-10-17 Thread Brian Freels-Stendel
Hello, I've done this a few times, and it's never been a problem for me. I make a backup of all of the .dat files in the log directory and the entire reports directory before deleting the .dat and .html files, just in case. B-- On 10/17/2011 at 1:14 AM, in message

[Dspace-tech] Regenerate monthly reports

2011-10-11 Thread Alan Orth
Hey, We noticed recently that our monthly stats hadn't run for the month of September. As it turns out, a batch import had imported some items with malformed `dc.date.accessioned` date fields, which was causing the stats scripts to die. We finally tracked down all the items with these bad