On Wed, Oct 19, 2011 at 03:55:27PM +0300, Alan Orth wrote: > Thanks for the replies... I just gave this a go. > > Wow. I had been zipping the historical log files in a cron job, so I > had to unzip them to do the log analysis. I had 25 gigs free, but ran > out of disk space unzipping them!
It would be nice if the stat. converters and loaders were able to run as filters, so you could do something like: gunzip < foo.log | dspace stats-log-converter | dspace stats-log-importer but no: ClassicDSpaceLogConverter requires named files. Would anyone have a problem with writing the status messages to System.err instead of System.out, so System.out is free for pipelining? > I didn't anticipate them taking that much space... can the log scripts, > by any chance, work on gzipped log files? Or, is there any way to do > this on another machine, if I recreate the environment perhaps? There's no decompression support that I can see. Maybe you can NFS or CIFS mount one machine to another? If you copy [DSpace]/bin and /lib and /config to a spare box, I think that might work (but *test* this, because I haven't!) I don't think the converter uses any external data except what you give it, and the importer should only need to know where Solr can be reached. However Solr would have to be listening on a socket that can be reached from the second machine. The simplest way is probably just to use the second machine as a storage server and do the work on the DSpace host. -- Mark H. Wood, Lead System Programmer [email protected] Asking whether markets are efficient is like asking whether people are smart.
pgpDGsfaRcmyd.pgp
Description: PGP signature
------------------------------------------------------------------------------ All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity and more. Splunk takes this data and makes sense of it. Business sense. IT sense. Common sense. http://p.sf.net/sfu/splunk-d2d-oct
_______________________________________________ DSpace-tech mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/dspace-tech
