rotatelogs is rotating too quickly...
I'm a newbie admin and I've just figure out something that will be obvious to most on this list... that apache log files can get big, fast. I did a df for another reason the other day and was surprised to see my /var partition at 85% full. Anyway, I did some googling and decided to use rotatelogs to do just that. Now... I've updated all my apache config files to use rotate logs, like this: ErrorLog |/usr/local/sbin/rotatelogs /var/log/www/nes.com-error.log. %Y-%m-%d-%H_%M_%S 86400 CustomLog |/usr/local/sbin/rotatelogs /var/log/www/nes.com- access.log.%Y-%m-%d-%H_%M_%S 5M combined Unless I am blind and not seeing something, this is right out of the man page. The problem is that the access logs are rolling over very quickly... no where near the 5 Meg I expected. It's not quite every access, but almost. -rw-r--r-- 1 root wheel 5147 Aug 20 08:19 nes.com-access.log. 2008-08-20-12_19_25 -rw-r--r-- 1 root wheel204 Aug 20 08:21 nes.com-access.log. 2008-08-20-12_20_58 -rw-r--r-- 1 root wheel195 Aug 20 08:24 nes.com-access.log. 2008-08-20-12_24_24 -rw-r--r-- 1 root wheel204 Aug 20 08:26 nes.com-access.log. 2008-08-20-12_26_15 -rw-r--r-- 1 root wheel181 Aug 20 08:29 nes.com-access.log. 2008-08-20-12_28_44 -rw-r--r-- 1 root wheel181 Aug 20 08:31 nes.com-access.log. 2008-08-20-12_31_35 -rw-r--r-- 1 root wheel204 Aug 20 08:34 nes.com-access.log. 2008-08-20-12_34_25 -rw-r--r-- 1 root wheel284 Aug 20 08:37 nes.com-access.log. 2008-08-20-12_36_56 -rw-r--r-- 1 root wheel710 Aug 20 08:45 nes.com-access.log. 2008-08-20-12_40_09 -rw-r--r-- 1 root wheel 3364 Aug 20 08:38 nes.com-error.log. 2008-08-20-00_00_00 I've been scratching my head over this for an hour, but can't see anything wrong. Any hints much appreciated. -- John ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: rotatelogs is rotating too quickly...
On Wed, Aug 20, 2008 at 2:58 PM, John Almberg [EMAIL PROTECTED] wrote: I'm a newbie admin and I've just figure out something that will be obvious to most on this list... that apache log files can get big, fast. I did a df for another reason the other day and was surprised to see my /var partition at 85% full. Anyway, I did some googling and decided to use rotatelogs to do just that. Now... I've updated all my apache config files to use rotate logs, like this: ErrorLog |/usr/local/sbin/rotatelogs /var/log/www/nes.com-error.log.%Y-%m-%d-%H_%M_%S 86400 CustomLog |/usr/local/sbin/rotatelogs /var/log/www/nes.com-access.log.%Y-%m-%d-%H_%M_%S 5M combined Unless I am blind and not seeing something, this is right out of the man page. The problem is that the access logs are rolling over very quickly... no where near the 5 Meg I expected. It's not quite every access, but almost. -rw-r--r-- 1 root wheel 5147 Aug 20 08:19 nes.com-access.log.2008-08-20-12_19_25 -rw-r--r-- 1 root wheel204 Aug 20 08:21 nes.com-access.log.2008-08-20-12_20_58 -rw-r--r-- 1 root wheel195 Aug 20 08:24 nes.com-access.log.2008-08-20-12_24_24 -rw-r--r-- 1 root wheel204 Aug 20 08:26 nes.com-access.log.2008-08-20-12_26_15 -rw-r--r-- 1 root wheel181 Aug 20 08:29 nes.com-access.log.2008-08-20-12_28_44 -rw-r--r-- 1 root wheel181 Aug 20 08:31 nes.com-access.log.2008-08-20-12_31_35 -rw-r--r-- 1 root wheel204 Aug 20 08:34 nes.com-access.log.2008-08-20-12_34_25 -rw-r--r-- 1 root wheel284 Aug 20 08:37 nes.com-access.log.2008-08-20-12_36_56 -rw-r--r-- 1 root wheel710 Aug 20 08:45 nes.com-access.log.2008-08-20-12_40_09 -rw-r--r-- 1 root wheel 3364 Aug 20 08:38 nes.com-error.log.2008-08-20-00_00_00 That does not look like 5 Meg but 5 Minutes. Riaan ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: rotatelogs is rotating too quickly...
That does not look like 5 Meg but 5 Minutes. I don't think so... From the man pages: CustomLog |bin/rotatelogs /var/logs/logfile 5M common This configuration will rotate the logfile whenever it reaches a size of 5 megabytes. ErrorLog |bin/rotatelogs /var/logs/errorlog.%Y-%m-%d-%H_ %M_%S 5M This configuration will rotate the error logfile whenever it reaches a size of 5 megabytes, and the suffix to the logfile name will be created of the form errorlog.-mm-dd-HH_MM_SS. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: rotatelogs is rotating too quickly...
On Wed, Aug 20, 2008 at 4:03 PM, John Almberg [EMAIL PROTECTED] wrote: That does not look like 5 Meg but 5 Minutes. I don't think so... From the man pages: CustomLog |bin/rotatelogs /var/logs/logfile 5M common This configuration will rotate the logfile whenever it reaches a size of 5 megabytes. ErrorLog |bin/rotatelogs /var/logs/errorlog.%Y-%m-%d-%H_%M_%S 5M This configuration will rotate the error logfile whenever it reaches a size of 5 megabytes, and the suffix to the logfile name will be created of the form errorlog.-mm-dd-HH_MM_SS. Oops sorry. Didn't look close. Initial inspection of log file times looked like 5 mins. Riaan. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: rotatelogs is rotating too quickly...
John Almberg wrote: I'm a newbie admin and I've just figure out something that will be obvious to most on this list... that apache log files can get big, fast. What apache version you are using? rotatelogs syntax differ a lot between them. -- Sphinx of black quartz judge my vow. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: rotatelogs is rotating too quickly...
In response to John Almberg [EMAIL PROTECTED]: I'm a newbie admin and I've just figure out something that will be obvious to most on this list... that apache log files can get big, fast. I did a df for another reason the other day and was surprised to see my /var partition at 85% full. Anyway, I did some googling and decided to use rotatelogs to do just that. Now... I've updated all my apache config files to use rotate logs, like this: ErrorLog |/usr/local/sbin/rotatelogs /var/log/www/nes.com-error.log. %Y-%m-%d-%H_%M_%S 86400 CustomLog |/usr/local/sbin/rotatelogs /var/log/www/nes.com- access.log.%Y-%m-%d-%H_%M_%S 5M combined Unless I am blind and not seeing something, this is right out of the man page. The problem is that the access logs are rolling over very quickly... no where near the 5 Meg I expected. It's not quite every access, but almost. -rw-r--r-- 1 root wheel 5147 Aug 20 08:19 nes.com-access.log. 2008-08-20-12_19_25 -rw-r--r-- 1 root wheel204 Aug 20 08:21 nes.com-access.log. 2008-08-20-12_20_58 -rw-r--r-- 1 root wheel195 Aug 20 08:24 nes.com-access.log. 2008-08-20-12_24_24 -rw-r--r-- 1 root wheel204 Aug 20 08:26 nes.com-access.log. 2008-08-20-12_26_15 -rw-r--r-- 1 root wheel181 Aug 20 08:29 nes.com-access.log. 2008-08-20-12_28_44 -rw-r--r-- 1 root wheel181 Aug 20 08:31 nes.com-access.log. 2008-08-20-12_31_35 -rw-r--r-- 1 root wheel204 Aug 20 08:34 nes.com-access.log. 2008-08-20-12_34_25 -rw-r--r-- 1 root wheel284 Aug 20 08:37 nes.com-access.log. 2008-08-20-12_36_56 -rw-r--r-- 1 root wheel710 Aug 20 08:45 nes.com-access.log. 2008-08-20-12_40_09 -rw-r--r-- 1 root wheel 3364 Aug 20 08:38 nes.com-error.log. 2008-08-20-00_00_00 I've been scratching my head over this for an hour, but can't see anything wrong. Any hints much appreciated. While your configuration certainly matches the examples in the docs, I can't imagine (for the life of me) how that would work. How would each httpd process know not to step on the toes of another when rotation time occurred? And what timestamp should it use when writing to the file -- the current one or the one when the process was started? Despite the fact that the docs say otherwise, the behaviour you're seeing is what I would _expect_. I hate to bring up this possibility, but perhaps the docs are _incorrect_! *blasphemy*!! Anyway, all ridiculous melodrama aside, I suspect that rotatelogs does perform as advertised, or (possibly) that the version of docs that you're reading does not match the version of rotatelogs that you're using. I know it's not a great answer, but it's the best I've got. Why not use the first example in the man page, as it seems much more likely to actually work in my opinion. -- Bill Moran http://www.potentialtech.com ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Fwd: rotatelogs is rotating too quickly...
John Almberg wrote: I'm a newbie admin and I've just figure out something that will be obvious to most on this list... that apache log files can get big, fast. What apache version you are using? rotatelogs syntax differ a lot between them. Version - Apache/2.2.6 (FreeBSD) mod_ssl/2.2.6 OpenSSL/0.9.7m DAV/2 PHP/5.2.5 with Suhosin-Patch rotatelogs doesn't seem to have a --version argument, but if I just type it on the command line it prints out the following usage message: [EMAIL PROTECTED]:www] rotatelogs Usage: rotatelogs [-l] logfile rotation time in seconds [offset minutes from UTC] or rotation size in megabytes Add this: TransferLog |rotatelogs /some/where 86400 or TransferLog |rotatelogs /some/where 5M to httpd.conf. The generated name will be /some/where. where is the system time at which the log nominally starts (N.B. if using a rotation time, the time will always be a multiple of the rotation time, so you can synchronize cron scripts with it). At the end of each rotation time or when the file size is reached a new log is started. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Fwd: rotatelogs is rotating too quickly...
On Aug 20, 2008, at 10:09 AM, Riaan Kruger wrote: On Wed, Aug 20, 2008 at 4:03 PM, John Almberg [EMAIL PROTECTED] wrote: That does not look like 5 Meg but 5 Minutes. I don't think so... From the man pages: CustomLog |bin/rotatelogs /var/logs/logfile 5M common This configuration will rotate the logfile whenever it reaches a size of 5 megabytes. ErrorLog |bin/rotatelogs /var/logs/errorlog.%Y-%m-%d-%H_ %M_%S 5M This configuration will rotate the error logfile whenever it reaches a size of 5 megabytes, and the suffix to the logfile name will be created of the form errorlog.-mm-dd-HH_MM_SS. Oops sorry. Didn't look close. Initial inspection of log file times looked like 5 mins. NP. That was my first thought, too, even though I had copied the line right out of the man pages. I had to read it again to make sure. Unfortunately, it's more complex than that... check out this list: ls -lt nes* -rw-r--r-- 1 root wheel 35846 Aug 20 10:19 nes.com-access.log. 2008-08-20-12_40_09 -rw-r--r-- 1 root wheel 10117 Aug 20 10:01 nes.com-access.log. 2008-08-20-13_56_42 -rw-r--r-- 1 root wheel284 Aug 20 08:37 nes.com-access.log. 2008-08-20-12_36_56 -rw-r--r-- 1 root wheel204 Aug 20 08:34 nes.com-access.log. 2008-08-20-12_34_25 -rw-r--r-- 1 root wheel181 Aug 20 08:31 nes.com-access.log. 2008-08-20-12_31_35 -rw-r--r-- 1 root wheel181 Aug 20 08:29 nes.com-access.log. 2008-08-20-12_28_44 -rw-r--r-- 1 root wheel204 Aug 20 08:26 nes.com-access.log. 2008-08-20-12_26_15 -rw-r--r-- 1 root wheel195 Aug 20 08:24 nes.com-access.log. 2008-08-20-12_24_24 -rw-r--r-- 1 root wheel204 Aug 20 08:21 nes.com-access.log. 2008-08-20-12_20_58 -rw-r--r-- 1 root wheel 5147 Aug 20 08:19 nes.com-access.log. 2008-08-20-12_19_25 Note the first two files on the list... the second file was created *after* the first one, but the first one is newer. That is because Apache is now logging to the first one, again. In other words, 1. it stopped logging to nes.com-access.log.2008-08-20-12_36_56, and created nes.com-access.log.2008-08-20-12_40_09 2. logged to that for awhile, then created nes.com-access.log. 2008-08-20-13_56_42 3. logged to that for awhile, and then *went back* to logging to nes.com-access.log.2008-08-20-12_40_09 I verified this by inspecting the two files and checking the times in the log. For example, these lines in nes.com-access.log. 2008-08-20-12_40_09 74.15.67.21 - - [20/Aug/2008:09:56:35 -0400] GET /favicon.ico HTTP/ 1.1 200 20 - Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727) 66.249.71.139 - - [20/Aug/2008:09:57:16 -0400] GET /browse_item.html? category_id=117item_id=1039page=12 HTTP/1.1 200 2806 - Mozilla/ 5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) And these lines in nes.com-access.log.2008-08-20-13_56_42 74.15.67.21 - - [20/Aug/2008:09:56:42 -0400] GET /payment/form/ HTTP/ 1.1 200 2348 http://www.newenglandstamp.com/; Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727) 74.15.67.21 - - [20/Aug/2008:09:56:42 -0400] GET /stylesheets/ public.css HTTP/1.1 200 1416 https://www.newenglandstamp.com/ payment/form/ Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727) Note that the time stamps in the second set are *between* the two lines in the first set. i.e., Apache was logging to one file, then switched to another, and then switched back... As I said, I'm a newbie, but this seems a bit odd to me... -- John Websites for On-line Collectible Dealers Identry, LLC John Almberg (631) 546-5079 [EMAIL PROTECTED] www.identry.com ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: rotatelogs is rotating too quickly...
Unfortunately, it's more complex than that... check out this list: ls -lt nes* -rw-r--r-- 1 root wheel 35846 Aug 20 10:19 nes.com-access.log. 2008-08-20-12_40_09 -rw-r--r-- 1 root wheel 10117 Aug 20 10:01 nes.com-access.log. 2008-08-20-13_56_42 -rw-r--r-- 1 root wheel284 Aug 20 08:37 nes.com-access.log. 2008-08-20-12_36_56 -rw-r--r-- 1 root wheel204 Aug 20 08:34 nes.com-access.log. 2008-08-20-12_34_25 -rw-r--r-- 1 root wheel181 Aug 20 08:31 nes.com-access.log. 2008-08-20-12_31_35 -rw-r--r-- 1 root wheel181 Aug 20 08:29 nes.com-access.log. 2008-08-20-12_28_44 -rw-r--r-- 1 root wheel204 Aug 20 08:26 nes.com-access.log. 2008-08-20-12_26_15 -rw-r--r-- 1 root wheel195 Aug 20 08:24 nes.com-access.log. 2008-08-20-12_24_24 -rw-r--r-- 1 root wheel204 Aug 20 08:21 nes.com-access.log. 2008-08-20-12_20_58 -rw-r--r-- 1 root wheel 5147 Aug 20 08:19 nes.com-access.log. 2008-08-20-12_19_25 Ah... I think I figured this out... Of course, I had to change a bunch of apache config files to switch from normal log files to piping logs through rotatelogs. I tested each config change after making it, to make sure I didn't mess something up. Of course I had to restart Apache each time I made a change. Here's the key: Apache creates a new access log for every site each time it restarts. So the list above reflects how fast it took me to update a config file and restart apache, not anything that rotate logs was doing. That makes sense, so I guess I can live with the 'makes new log file on restart' quirk. Thanks for the help! -- John ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
