--- Jeff Nokes <[EMAIL PROTECTED]> wrote: > > --- Aengus <[EMAIL PROTECTED]> wrote: > > On Thursday, April 07, 2005 7:28 PM [GMT], > > Jeff Nokes <[EMAIL PROTECTED]> wrote: > > > > > Hi, > > > I've been using Analog 5.2 on RedHat 7.2 (2.4 kernel / ext3) for over > > > a year now, and everything has been working fine, until I tried to > > > process my March 2005 log files. I have multiple hosts that support > > > my sites, and I always merge all log files into one large log file, > > > and gzip it. I then process the one large file with analog to get my > > > results. Up through Feb 2005, the compressed large file has been > > > growing to about 2.8 GB in size, but still analog has been working > > > fine with it. Now, with my March 2005 compressed file, it is over 3 > > > GB (3241590586) to be exact, and analog is now saying it can't find > > > my file to process it. Nothing has changed with the way I process > > > the files, except for the file size of the file, and of course the > > > file name, which has never been an issue in the past. > > > > > > $ ls -l > > > /var/row/prod/reports/logfiles/200503.farm.wc.dm.row.access.log.gz > > > > > > -rw-rw-r-- 1 rowroot rowroot 3241590586 Apr 6 14:30 > > > /var/row/prod/reports/logfiles/200503.farm.w > > > c.dm.row.access.log.gz > > > > > > It's there, as I know it is. Anyone have any ideas of what may be > > > going on? Thanks, > > > > 3G is an unusual place for a change in behaviour to occur, but there's a > > post in the archives that refers to a 3G limit on Linux: > > http://www.mail-archive.com/[email protected]/msg00169.html > > > > Have you tried putting the logs in two .gz files? Analog is just as > > happy with 2 input files as 1. > > > > Aengus > > > Thanks for the quick reply Aengus. I looked at the link you provided, and > it > seems to be discussing the addressable RAM for 32-bit systems, not file > size. > My system as 4GB of usable RAM, so I have plenty of room. Also, with the > ext2 > file system, a single inode can handle up to 16GB in size if I recall > correctly, so I would imagine ext3 can do at least that, if not more, but > I'm > not sure. Anyway, if Linux couldn't handle the file size, I would have > thought it would have complained when I created that huge file, or at least > had a corrupted inode, but it doesn't, so I'm assuming the OS is OK with it. > > I'll go ahead and try splitting that larger file into two files and see what > happens, and post my results. > > Thanks, > - Jeff
Just wanted to send a note back saying that splitting the files worked fine. Interesting that analog has a file size barrier when Linux doesn't. Maybe it's time to upgrade to 6.0. Can I still use ReportMagic with 6.0 without changing any configurations for RM? Thanks for the suggestion to split the data files. - Jeff +------------------------------------------------------------------------ | TO UNSUBSCRIBE from this list: | http://lists.meer.net/mailman/listinfo/analog-help | | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general | List archives: http://www.analog.cx/docs/mailing.html#listarchives +------------------------------------------------------------------------
