[ catching up on lists ]
One thing I’d strongly suggest is decoupling rotation from compression. If you
compress from logrotate, it compresses files as it rotates them. This can add
a significant amount of time to the logrotate job. if it happens significantly
differently night to night, it might mess up your “daily” rotations.
This also makes it easy to rotate everything at midnight so that the files get
the right names for the days containing the data, modulo a couple minutes’
worth. IIRC I do the rotation at 23:59; if you have a truly large number of
files o rotate you might need another minute.
Compression and/or deletion later on is pretty simple with a find piped to
xargs.
--
Dan Pritts
System & Security Architect
ICPSR Computing and Network Services
On 14 Nov 2023, at 6:03, Dimi Onobodies via rsyslog wrote:
> So I added "-v" option on the cronjob and redirected output to a file. I
> observed the following:
>
> considering log /data/servers/rsyslog/ldap-access.log
> log needs rotating
>
> rotating log /data/logs/apache.log, log->rotateCount is 4
> dateext suffix '-20231114'
> glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
> glob finding old rotated logs failed
>
> rotating log /data/logs/maillog, log->rotateCount is 4
> dateext suffix '-20231114'
> glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
> destination /data/logs/maillog-20231114.gz already exists, skipping rotation
>
>
> All templated syslog files get "glob finding old rotated logs failed". Thing
> is, I can see the file present in the filesystem:
>
> # ls /data/logs/apache.log-20231114
> /data/logs/apache.log-20231114
>
> However, when I tried to view the logrotate file I noticed that it has
> different output:
>
> "considering log /data/logs/apache.log
> log does not need rotating (log has been rotated at 2023-11-14 0:1, that is
> not day ago yet)"
>
> Its like logrotate was executed again. Also, there are two logrotate messages
> in system log
>
> # grep ALERT /var/log/messages
> 2023-11-14T01:01:06.664171-08:00 ALERT exited abnormally with [1]
> 2023-11-14T01:24:56.322123-08:00 ALERT exited abnormally with [1]
>
>
> One more thing I noticed when i sshed on the system was that gzip was running:
>
>
> # stat /data/logs/logrotate_debug.txt
> File: ‘/data/servers/logrotate_debug.txt’
> Size: 14410 Blocks: 32 IO Block: 4096 regular file
> Device: 810h/2064d Inode: 36438030 Links: 1
> Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
> Access: 2023-11-13 07:26:25.868304144 -0800
> Modify: 2023-11-14 01:01:06.616954323 -0800
> Change: 2023-11-14 01:01:06.616954323 -0800
> Birth: -
>
>
> # ps -ef|grep gz
> root 1313 17221 97 00:45 ? 00:19:29 /bin/gzip
> root 3849 1392 0 01:05 pts/1 00:00:00 grep --color=auto gz
>
> # ps -ef|grep 17221
> root 1313 17221 97 00:45 ? 00:19:46 /bin/gzip
> root 4108 1392 0 01:05 pts/1 00:00:00 grep --color=auto 17221
> root 17221 17219 0 00:01 ? 00:00:05 /usr/sbin/logrotate -v -s
> /var/lib/logrotate.status /etc/logrotate.conf
>
> # ps -ef|grep logrotate
> root 4750 1392 0 01:06 pts/1 00:00:00 grep --color=auto logrotate
> root 17219 17197 0 00:01 ? 00:00:00 /bin/sh
> /etc/cron.hourly/logrotate
> root 17220 17197 0 00:01 ? 00:00:00 awk -v
> progname=/etc/cron.hourly/logrotate progname { ???? print progname ":\n"
> ???? progname=""; ??? } ??? { print; }
> root 17221 17219 0 00:01 ? 00:00:05 /usr/sbin/logrotate -v -s
> /var/lib/logrotate.status /etc/logrotate.conf
>
>
> Still cannot pinpoint if this is a rsyslog, logrotate, gzip issue. I dont get
> it why this is happening only on logs generated with template configuration
> directives.
>
> D.
>
> ________________________________
> From: rsyslog <[email protected]> on behalf of Mariusz Kruk
> via rsyslog <[email protected]>
> Sent: Monday, November 13, 2023 1:32 PM
> To: [email protected] <[email protected]>
> Cc: Mariusz Kruk <[email protected]>
> Subject: Re: [rsyslog] logrotate wont compress-rotate syslog templated files
>
> If the logrotate daily invocation already happened today, it won't
> consider the files "rotate-worthy" before next day.
>
> So you might either add "-v" to the cron job so that it will tell you
> verbosely what/why it does on the next "proper" run. Or do "-f -d".
>
> MK
>
> On 13.11.2023 14:13, Dimi Onobodies via rsyslog wrote:
>> Hi Maiusz
>>
>> Forgot to add that in my description. I already tried that and i didnt see
>> any error reporte from logrotate:
>>
>>
>> * logrotate -d -v -s /var/lib/logrotate.status /etc/logrotate.conf
>>
>> considering log /data/logs/apache.log
>> log does not need rotating (log has been already rotated)
>>
>>
>> I am thinking to force (-f) rotation and see if this will show anything
>> ________________________________
>> From: rsyslog <[email protected]> on behalf of Mariusz Kruk
>> via rsyslog <[email protected]>
>> Sent: Monday, November 13, 2023 9:13 AM
>> To: [email protected] <[email protected]>
>> Cc: Mariusz Kruk <[email protected]>
>> Subject: Re: [rsyslog] logrotate wont compress-rotate syslog templated files
>>
>> First and foremost, you should check what and why logrotate does when
>> rotating files (the -v and - d options for logrotate). That will tell
>> you what is actually happening when the rotation should occur.
>>
>>
>> On 13.11.2023 09:57, Dimi Onobodies via rsyslog wrote:
>>> Hi, I am facing a weird logrotate issue in my central syslog system
>>> (receiving logs from various host via imtcp). Logrotate does not compress
>>> syslog templated log files. It does compress and rotates other log files
>>> generated by rsyslog, but not the ones generated using a template.
>> _______________________________________________
>> rsyslog mailing list
>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
>> LIKE THAT.
>> _______________________________________________
>> rsyslog mailing list
>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
>> LIKE THAT.
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
> LIKE THAT.
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
> LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
