--- Aengus <[EMAIL PROTECTED]> wrote: > On Thursday, April 07, 2005 7:28 PM [GMT], > Jeff Nokes <[EMAIL PROTECTED]> wrote: > > > Hi, > > I've been using Analog 5.2 on RedHat 7.2 (2.4 kernel / ext3) for over > > a year now, and everything has been working fine, until I tried to > > process my March 2005 log files. I have multiple hosts that support > > my sites, and I always merge all log files into one large log file, > > and gzip it. I then process the one large file with analog to get my > > results. Up through Feb 2005, the compressed large file has been > > growing to about 2.8 GB in size, but still analog has been working > > fine with it. Now, with my March 2005 compressed file, it is over 3 > > GB (3241590586) to be exact, and analog is now saying it can't find > > my file to process it. Nothing has changed with the way I process > > the files, except for the file size of the file, and of course the > > file name, which has never been an issue in the past. > > > > $ ls -l > > /var/row/prod/reports/logfiles/200503.farm.wc.dm.row.access.log.gz > > > > -rw-rw-r-- 1 rowroot rowroot 3241590586 Apr 6 14:30 > > /var/row/prod/reports/logfiles/200503.farm.w > > c.dm.row.access.log.gz > > > > It's there, as I know it is. Anyone have any ideas of what may be > > going on? Thanks, > > 3G is an unusual place for a change in behaviour to occur, but there's a > post in the archives that refers to a 3G limit on Linux: > http://www.mail-archive.com/[email protected]/msg00169.html > > Have you tried putting the logs in two .gz files? Analog is just as > happy with 2 input files as 1. > > Aengus
Thanks for the quick reply Aengus. I looked at the link you provided, and it seems to be discussing the addressable RAM for 32-bit systems, not file size. My system as 4GB of usable RAM, so I have plenty of room. Also, with the ext2 file system, a single inode can handle up to 16GB in size if I recall correctly, so I would imagine ext3 can do at least that, if not more, but I'm not sure. Anyway, if Linux couldn't handle the file size, I would have thought it would have complained when I created that huge file, or at least had a corrupted inode, but it doesn't, so I'm assuming the OS is OK with it. I'll go ahead and try splitting that larger file into two files and see what happens, and post my results. Thanks, - Jeff +------------------------------------------------------------------------ | TO UNSUBSCRIBE from this list: | http://lists.meer.net/mailman/listinfo/analog-help | | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general | List archives: http://www.analog.cx/docs/mailing.html#listarchives +------------------------------------------------------------------------
