On Tue, Jun 25, 2013 at 11:51 AM, David Lang <[email protected]> wrote: > On Tue, 25 Jun 2013, Xuri Nagarin wrote: > > Yes, a manual HUP released them all but not instantaneously. Took a few >> seconds. >> > > given how many files you have open, It's not too surprising that it takes > a little time > > A logrotate bug? >> > > double check the contents (and timestamp) of /var/run/syslog.pid, it may > not be what you are expecting >
/var/run/syslog.pid contents reflect the right PID number. What's more funny is that if I run "watch -d 'sudo lsof | grep deleted | grep rsyslog'", I can see the deleted log files growing in size. Logrotate should never create a logfile.1 since rotate is set to 0. Right? > > since the HUP does free the files, my suspicion is that rsyslog is not > getting the HUP in the first place. This would point me at logrotate, or > the way that logrotate is getting the PID > Going to run logrotate with strace and see what shows up. Thanks. > > David Lang > > > > On Tue, Jun 25, 2013 at 11:38 AM, David Lang <[email protected]> wrote: >> >> If you manually do a kill -HUP 11311 do these files get released? >>> >>> >>> David Lang >>> >>> On Tue, 25 Jun 2013, Xuri Nagarin wrote: >>> >>> More fun info: >>> >>>> >>>> $ sudo lsof -s | awk '$5 == "REG"' | sort -n -r -k 7,7 | head -n 50 | >>>> grep >>>> deleted >>>> rsyslogd 11331 root 20w REG 9,2 >>>> 117175430149 >>>> 8847368 /var/log/joe/CISCO/app7/****logfile.1 (deleted) >>>> >>>> rsyslogd 11331 root 9w REG 9,2 53533745904 >>>> 8839187 /var/log/joe/Microsoft/app1/****logfile.1 (deleted) >>>> >>>> rsyslogd 11331 root 10w REG 9,2 >>>> 16268001760 >>>> 8839237 /var/log/joe/Microsoft/app2/****logfile.1 (deleted) >>>> >>>> rsyslogd 11331 root 18w REG 9,2 1831944964 >>>> 8847369 /var/log/joe/Microsoft/app3/****logfile.1 (deleted) >>>> >>>> >>>> >>>> >>>> >>>> On Tue, Jun 25, 2013 at 12:10 PM, Xuri Nagarin <[email protected]> >>>> wrote: >>>> >>>> Happening again: >>>> >>>>> >>>>> $ sudo du -sh * >>>>> 9.8M bin >>>>> 26M boot >>>>> 220K dev >>>>> 6.9M etc >>>>> 3.5G home >>>>> 129M lib >>>>> 22M lib64 >>>>> 20K lost+found >>>>> 8.0K media >>>>> 8.0K mnt >>>>> 6.4G opt >>>>> du: cannot access `proc/2713/task/9576': No such file or directory >>>>> du: cannot access `proc/9490/task/9490/fd/4': No such file or directory >>>>> du: cannot access `proc/9490/task/9490/fdinfo/4'****: No such file or >>>>> >>>>> directory >>>>> du: cannot access `proc/9490/fd/4': No such file or directory >>>>> du: cannot access `proc/9490/fdinfo/4': No such file or directory >>>>> 0 proc >>>>> 116K root >>>>> 13M sbin >>>>> 4.0K selinux >>>>> 4.0K srv >>>>> 0 sys >>>>> 296K tmp >>>>> 1.3G usr >>>>> 39G var >>>>> >>>>> $ sudo df -kh >>>>> Filesystem Size Used Avail Use% Mounted on >>>>> /dev/md2 913G 212G 655G 25% / >>>>> tmpfs 16G 0 16G 0% /dev/shm >>>>> /dev/md0 97M 31M 62M 34% /boot >>>>> >>>>> I am seeing the same baheviour on a second system. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Tue, Jun 25, 2013 at 12:04 PM, Xuri Nagarin <[email protected]> >>>>> wrote: >>>>> >>>>> My suspicion was something to do with dynamic file creation but >>>>> looking >>>>> >>>>>> at the code, both static and dynamic file creation use the same >>>>>> function of >>>>>> file "open": >>>>>> >>>>>> From tools/omfile.c >>>>>> ------------xxxxxxxxxxxxxxxxx-****-------------- >>>>>> fd = open((char*) newFileName, >>>>>> O_WRONLY|O_APPEND|O_CREAT|O_****NOCTTY|O_CLOEXEC, >>>>>> 556 pData->fCreateMode); >>>>>> ------------xxxxxxxxxxxxxxxxx-****-------------- >>>>>> >>>>>> >>>>>> "O_APPEND" should take care of file being rotated while rsyslog is >>>>>> trying >>>>>> to write to it. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Jun 25, 2013 at 11:47 AM, Xuri Nagarin <[email protected]> >>>>>> wrote: >>>>>> >>>>>> Not a typo or error, in RHEL the rsyslogd start up script in init.d >>>>>> has >>>>>> >>>>>>> an explicit variable "PIDFILE" set as: >>>>>>> PIDFILE=/var/run/syslogd.pid >>>>>>> >>>>>>> :) >>>>>>> >>>>>>> I did not write the script, using whatever was bundled in the RPM I >>>>>>> grabbed from Adiscon. >>>>>>> >>>>>>> To confirm: >>>>>>> $ ps ax | grep rsyslog >>>>>>> 11331 ? Sl 61:18 /sbin/rsyslogd -i /var/run/syslogd.pid -x >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, Jun 25, 2013 at 11:41 AM, Soham Chakraborty < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>>> >>>>>>>> In the logrotate config, change it to /var/run/rsyslog.pid and test. >>>>>>>> On >>>>>>>> a >>>>>>>> quick skim, it looks like a typo. >>>>>>>> >>>>>>>> Soham >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Jun 25, 2013 at 11:07 PM, David Lang <[email protected]> wrote: >>>>>>>> >>>>>>>> well, one thing that looks wrong is that logrotate is looking for >>>>>>>> >>>>>>>>> /var/run/syslog.pid, but on my systems the pid is in >>>>>>>>> >>>>>>>>> /var/log/rsyslog.pid >>>>>>>> >>>>>>>> >>>>>>>>> are you sure that rsyslog is actually getting the HUP? >>>>>>>>> >>>>>>>>> can you try sending it a HUP manually and see if it closes the >>>>>>>>> files? >>>>>>>>> >>>>>>>>> David Lang >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tue, 25 Jun 2013, Xuri Nagarin wrote: >>>>>>>>> >>>>>>>>> Date: Tue, 25 Jun 2013 11:34:08 -0700 >>>>>>>>> >>>>>>>>> From: Xuri Nagarin <[email protected]> >>>>>>>>>> Reply-To: rsyslog-users <[email protected]> >>>>>>>>>> To: rsyslog-users <[email protected]> >>>>>>>>>> Subject: Re: [rsyslog] HUP-ing rsyslog does not free up disk space >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hi David, >>>>>>>>>> >>>>>>>>>> The master conf file is simple with few additions that are >>>>>>>>>> self-explanatory. The other conf in rsyslog.d, I have added >>>>>>>>>> comments >>>>>>>>>> inline >>>>>>>>>> in the conf that explain the config logic. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> /etc/rsyslog.conf >>>>>>>>>> ==============================******==========================** >>>>>>>>>> ==** >>>>>>>>>> >>>>>>>>>> ==** >>>>>>>>>> =========== >>>>>>>>>> module(load="impstats" interval="600" severity="7" >>>>>>>>>> >>>>>>>>>> log.syslog="off" /* need to turn log stream logging off! */ >>>>>>>>>> log.file="/var/log/rsyslog-******stats.log") >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> $ModLoad imuxsock.so # Unix sockets >>>>>>>>>> $ModLoad imklog.so # Kernel logger >>>>>>>>>> $MainMsgQueueSize 1000000 >>>>>>>>>> $OMFileIOBufferSize 512k >>>>>>>>>> $MaxMessageSize 8k >>>>>>>>>> $MainMsgQueueWorkerThreads 64 >>>>>>>>>> $umask 0000 >>>>>>>>>> $FileOwner joe >>>>>>>>>> $FileGroup joe >>>>>>>>>> $DirOwner joe >>>>>>>>>> $DirGroup joe >>>>>>>>>> $DirCreateMode 0755 >>>>>>>>>> $FileCreateMode 0644 >>>>>>>>>> >>>>>>>>>> $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat >>>>>>>>>> $IncludeConfig /etc/rsyslog.d/*.conf >>>>>>>>>> >>>>>>>>>> $RuleSet local >>>>>>>>>> *.info;mail.none;authpriv.******none;cron.none >>>>>>>>>> /var/log/messages >>>>>>>>>> authpriv.* >>>>>>>>>> >>>>>>>>>> /var/log/secure >>>>>>>>> >>>>>>>> >>>>>>>> mail.* >>>>>>>>> >>>>>>>>>> >>>>>>>>>> -/var/log/maillog >>>>>>>>> >>>>>>>> >>>>>>>> cron.* >>>>>>>>> /var/log/cron >>>>>>>>> >>>>>>>>>> *.emerg * >>>>>>>>>> uucp,news.crit >>>>>>>>>> >>>>>>>>>> /var/log/spooler >>>>>>>>> >>>>>>>> >>>>>>>> local7.* >>>>>>>>> >>>>>>>>>> >>>>>>>>>> /var/log/boot.log >>>>>>>>> >>>>>>>> >>>>>>>> $DefaultRuleset local >>>>>>>>> >>>>>>>>>> ==============================******==========================** >>>>>>>>>> ==** >>>>>>>>>> ==** >>>>>>>>>> =========== >>>>>>>>>> >>>>>>>>>> /etc/rsyslog.d/cef.conf >>>>>>>>>> ==============================******==========================** >>>>>>>>>> ==** >>>>>>>>>> >>>>>>>>>> ==** >>>>>>>>>> =========== >>>>>>>>>> # Senders are four Arcsight Logger devices that send logs in CEF >>>>>>>>>> >>>>>>>>>> format >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> #template for writing CEF formatted logs >>>>>>>>>> template(name="cefdynfile" type="string" >>>>>>>>>> string="/var/log/joe/%$!******vendor%/%$!product%/logfile") >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> #template for writing logs from non-CEF sources >>>>>>>>>> template(name="noncefdynfile" type="string" >>>>>>>>>> string="/var/log/joe/noncef/%******hostname%/%programname%/****** >>>>>>>>>> >>>>>>>>>> logfile") >>>>>>>>>> >>>>>>>>>> ruleset(name="tcpcef") { >>>>>>>>>> >>>>>>>>>> #CEF uses the pipe delimiter, fields 2 and 3 are product vendor >>>>>>>>>> and >>>>>>>>>> product >>>>>>>>>> type respectively >>>>>>>>>> set $!vendor = field($msg, 124, 2); >>>>>>>>>> set $!product = field($msg, 124, 3); >>>>>>>>>> >>>>>>>>>> # Rules to write CEF formatted logs to disk and send logs by app >>>>>>>>>> >>>>>>>>>> type to >>>>>>>>> >>>>>>>> >>>>>>>> their flume destinations that are listening on the same box >>>>>>>>> >>>>>>>>>> >>>>>>>>>> if $syslogtag=="CEF:" then { action (type="omfile" >>>>>>>>>> ASyncWriting="on" >>>>>>>>>> IOBufferSize="8192K" FileOwner="joe" FileGroup="joe" >>>>>>>>>> DirOwner="joe" >>>>>>>>>> DirGroup="joe" DirCreateMode="0755" FileCreateMode="0644" >>>>>>>>>> DynaFile="cefdynfile") } >>>>>>>>>> if $!product == "app1" then { action (type="omfwd" >>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> Port="5161" Protocol="tcp") stop } >>>>>>>>>> if $!product == "app2" then { action (type="omfwd" >>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> Port="5146" Protocol="tcp") stop } >>>>>>>>>> if $!product == "app3" then { action (type="omfwd" >>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> Port="5172" Protocol="tcp") stop } >>>>>>>>>> if $!product == "app4" then { action (type="omfwd" >>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> Port="5162" Protocol="tcp") stop } >>>>>>>>>> if $!product == "app5" then { action (type="omfwd" >>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> Port="5166" Protocol="tcp") stop } >>>>>>>>>> if $!product == "app6" then { action (type="omfwd" >>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> Port="5163" Protocol="tcp") stop } >>>>>>>>>> if $!product == "app7" then { action (type="omfwd" >>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> Port="5164" Protocol="tcp") stop } >>>>>>>>>> if $!product == "app8" then { action (type="omfwd" >>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> Port="5177" Protocol="tcp") stop } >>>>>>>>>> if $!product == "app9" then { action (type="omfwd" >>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> Port="5144" Protocol="tcp") stop } >>>>>>>>>> if $!product == "app10" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5145" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> if $!product == "app11" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5148" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> if $!product == "app12" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5180" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> if $!product == "app13" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5147" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> if $!product == "app14" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5149" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> if $!product == "app15" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5150" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> if $!product == "app16" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5151" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> if $!product == "app17" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5152" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> if $!product == "app18" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5153" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> if $!product == "app19" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5155" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> if $!product == "app20" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5156" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> if $!product == "app21" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5157" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> if $!product == "app22" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5158" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> if $!product == "app23" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5159" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> if $!product == "app24" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5160" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> if $!product == "app25" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5178" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> if $!product == "app26" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5165" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> if $!product == "app27" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5165" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> if $!product == "app28" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5167" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> if $!product == "app29" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5167" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> if $!product == "app30" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5179" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> if $!product == "app31" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5169" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> if $!product == "app32" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5170" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> if $!product == "app33" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5171" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> if $!product == "app34" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5174" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> if $!product == "app35" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5173" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> if $!product == "app36" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5175" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> if $!product == "app37" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5176" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> if $!product == "app38" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5154" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> if $!product == "app39" then { action (type="omfwd" >>>>>>>>>> >>>>>>>>>> Target="127.0.0.1" >>>>>>>>> >>>>>>>> >>>>>>>> Port="5181" Protocol="tcp") stop } >>>>>>>>> >>>>>>>>>> >>>>>>>>>> # Unfortunately, the four Arcsight Loggers also send us garbage so >>>>>>>>>> whatever >>>>>>>>>> could not be parsed/classified correctly by the rules above, gets >>>>>>>>>> >>>>>>>>>> parked >>>>>>>>> >>>>>>>> >>>>>>>> in >>>>>>>>> >>>>>>>>>> a catchall file. >>>>>>>>>> >>>>>>>>>> if $fromhost-ip == '10.1.1.100' or $fromhost-ip == '10.1.1.101' or >>>>>>>>>> $fromhost-ip == '10.1.1.102' or $fromhost-ip == '10.1.1.103' then >>>>>>>>>> { >>>>>>>>>> >>>>>>>>>> action >>>>>>>>> >>>>>>>> >>>>>>>> (type="omfile" FileOwner="joe" FileGroup="joe" DirOwner="joe" >>>>>>>>> >>>>>>>>>> DirGroup="joe" DirCreateMode="0755" FileCreateMode="0644" >>>>>>>>>> file="/var/log/joe/fallback/******logfile") >>>>>>>>>> >>>>>>>>>> & action (type="omfwd" Target="127.0.0.1" Port="5182" >>>>>>>>>> >>>>>>>>>> Protocol="tcp" >>>>>>>>> >>>>>>>> >>>>>>>> ) >>>>>>>>> >>>>>>>>>> stop } >>>>>>>>>> >>>>>>>>>> # Take care of all the non-CEF / BSD Syslog formatted streams >>>>>>>>>> coming >>>>>>>>>> >>>>>>>>>> in >>>>>>>>> >>>>>>>> >>>>>>>> else { >>>>>>>>> >>>>>>>>>> action (type="omfile" ASyncWriting="on" IOBufferSize="8192K" >>>>>>>>>> FileOwner="joe" FileGroup="joe" DirOwner="joe" DirGroup="joe" >>>>>>>>>> DirCreateMode="0755" FileCreateMode="0644" >>>>>>>>>> DynaFile="noncefdynfile") >>>>>>>>>> & action (type="omfwd" Target="127.0.0.1" Port="5182" >>>>>>>>>> >>>>>>>>>> Protocol="tcp") >>>>>>>>> >>>>>>>> >>>>>>>> } >>>>>>>>> >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> module(load="imtcp" ) # needs to be done just once >>>>>>>>>> input(type="imtcp" port="514" ruleset="tcpcef") >>>>>>>>>> ==============================******==========================** >>>>>>>>>> ==** >>>>>>>>>> >>>>>>>>>> ==** >>>>>>>>>> =========== >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Here's the logrotate code: >>>>>>>>>> ==============================******==========================** >>>>>>>>>> ==** >>>>>>>>>> >>>>>>>>>> ==** >>>>>>>>>> =========== >>>>>>>>>> /var/log/joe/*/*/* >>>>>>>>>> /var/log/joe/*/*/*/* >>>>>>>>>> { >>>>>>>>>> missingok >>>>>>>>>> size 1G >>>>>>>>>> rotate 0 >>>>>>>>>> sharedscripts >>>>>>>>>> postrotate >>>>>>>>>> if [ -f /var/run/syslog.pid ]; then \ >>>>>>>>>> kill -HUP `cat /var/run/syslog.pid`; \ >>>>>>>>>> fi; >>>>>>>>>> endscript >>>>>>>>>> } >>>>>>>>>> ==============================******==========================** >>>>>>>>>> ==** >>>>>>>>>> >>>>>>>>>> ==** >>>>>>>>>> =========== >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tue, Jun 25, 2013 at 10:12 AM, David Lang <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> On Tue, 25 Jun 2013, Xuri Nagarin wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On RHEL 6.2 64-bit, I have Rsyslog 7.4.1 (actually, the issue >>>>>>>>>>> has >>>>>>>>>>> existed >>>>>>>>>>> >>>>>>>>>>> for earlier versions too). >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> After logrotation, the logrotate script HUPs the rsyslogd pid >>>>>>>>>>>> but >>>>>>>>>>>> >>>>>>>>>>>> the >>>>>>>>>>> >>>>>>>>>> >>>>>>>> disk >>>>>>>>> >>>>>>>>>> space doesn't free up until I restart rsyslog. After a few hours >>>>>>>>>>>> >>>>>>>>>>>> "df" >>>>>>>>>>> >>>>>>>>>> >>>>>>>> reports a full filesystem whereas "ls" shows much smaller file >>>>>>>>> >>>>>>>>>> >>>>>>>>>>>> sizes. In >>>>>>>>>>> >>>>>>>>>> >>>>>>>> between this confusion, rsyslog seems to stop receiving log >>>>>>>>> >>>>>>>>>> >>>>>>>>>>>> streams. >>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>>> Is there a way for rsyslog to switch file handles when it is >>>>>>>>>>>> >>>>>>>>>>>> HUP-ed? >>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>>>> rsyslog is already supposed to close and re-open files when >>>>>>>>>>>> it's >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> HUP-ed, >>>>>>>>>> >>>>>>>>> >>>>>>>> and it seems to be working for me and many others. >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> could you send your config file so we can see if there is >>>>>>>>>>> anything >>>>>>>>>>> unusual >>>>>>>>>>> in it? >>>>>>>>>>> >>>>>>>>>>> David Lang >>>>>>>>>>> ______________________________********_________________ >>>>>>>>>>> rsyslog mailing list >>>>>>>>>>> http://lists.adiscon.net/********mailman/listinfo/rsyslog<http://lists.adiscon.net/******mailman/listinfo/rsyslog> >>>>>>>>>>> <http**://lists.adiscon.net/******mailman/listinfo/rsyslog<http://lists.adiscon.net/****mailman/listinfo/rsyslog> >>>>>>>>>>> > >>>>>>>>>>> < >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> http://lists.adiscon.net/******mailman/listinfo/rsyslog<http://lists.adiscon.net/****mailman/listinfo/rsyslog> >>>>>>>>>> <http:**//lists.adiscon.net/**mailman/**listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>>>>>>>>> > >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> <http:**//lists.adiscon.net/****mailman/**listinfo/rsyslog<http://lists.adiscon.net/**mailman/**listinfo/rsyslog> >>>>>>>>> <htt**p://lists.adiscon.net/mailman/****listinfo/rsyslog<http://lists.adiscon.net/mailman/**listinfo/rsyslog> >>>>>>>>> > >>>>>>>>> >>>>>>>>>> < >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> http://lists.adiscon.net/****mailman/listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>>>>>>>>> <http:**//lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>>>>>>>>> > >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> http://www.rsyslog.com/********professional-services/<http://www.rsyslog.com/******professional-services/> >>>>>>>>>>>> <http://**www.rsyslog.com/******professional-services/<http://www.rsyslog.com/****professional-services/> >>>>>>>>>>>> > >>>>>>>>>>>> >>>>>>>>>>> < >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> http://www.rsyslog.com/******professional-services/<http://www.rsyslog.com/****professional-services/> >>>>>>>>>> <http://**www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>>>>>>>> > >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> <http://**www.rsyslog.com/****professional-**services/<http://www.rsyslog.com/**professional-**services/> >>>>>>>>> <http:**//www.rsyslog.com/**professional-**services/<http://www.rsyslog.com/professional-**services/> >>>>>>>>> > >>>>>>>>> >>>>>>>>>> < >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> http://www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>>>>>>>> <http://**www.rsyslog.com/professional-**services/<http://www.rsyslog.com/professional-services/> >>>>>>>>>> > >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>>>>>> >>>>>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>>>>>> >>>>>>>>>>> myriad >>>>>>>>>> >>>>>>>>> >>>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> you >>>>>>>>>> >>>>>>>>> >>>>>>>> DON'T LIKE THAT. >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> ______________________________******_________________ >>>>>>>>>>> >>>>>>>>>>> rsyslog mailing list >>>>>>>>>> http://lists.adiscon.net/******mailman/listinfo/rsyslog<http://lists.adiscon.net/****mailman/listinfo/rsyslog> >>>>>>>>>> <http:**//lists.adiscon.net/**mailman/**listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>>>>>>>>> > >>>>>>>>>> < >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> http://lists.adiscon.net/****mailman/listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>>>>>>>> <http:**//lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>>>>>>>> > >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> http://www.rsyslog.com/******professional-services/<http://www.rsyslog.com/****professional-services/> >>>>>>>>> <http://**www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>>>>>>> > >>>>>>>>> >>>>>>>>>> < >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> http://www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>>>>>>> <http://**www.rsyslog.com/professional-**services/<http://www.rsyslog.com/professional-services/> >>>>>>>>> > >>>>>>>>> >>>>>>>>> >>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>>> >>>>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>>>>> >>>>>>>>>> myriad >>>>>>>>> >>>>>>>> >>>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >>>>>>>>> >>>>>>>>>> >>>>>>>>>> you >>>>>>>>> >>>>>>>> >>>>>>>> DON'T LIKE THAT. >>>>>>>>> >>>>>>>>>> >>>>>>>>>> ______________________________******_________________ >>>>>>>>>> >>>>>>>>>> rsyslog mailing list >>>>>>>>> http://lists.adiscon.net/******mailman/listinfo/rsyslog<http://lists.adiscon.net/****mailman/listinfo/rsyslog> >>>>>>>>> <http:**//lists.adiscon.net/**mailman/**listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>>>>>>>> > >>>>>>>>> < >>>>>>>>> >>>>>>>>> >>>>>>>>> http://lists.adiscon.net/****mailman/listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>>>>>>> <http:**//lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>>>>>>> > >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> http://www.rsyslog.com/******professional-services/<http://www.rsyslog.com/****professional-services/> >>>>>>>>> <http://**www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>>>>>>> > >>>>>>>>> < >>>>>>>>> >>>>>>>>> >>>>>>>>> http://www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>>>>>> <http://**www.rsyslog.com/professional-**services/<http://www.rsyslog.com/professional-services/> >>>>>>>> > >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>>>> >>>>>>>>> myriad >>>>>>>> >>>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >>>>>>>>> you >>>>>>>>> DON'T LIKE THAT. >>>>>>>>> >>>>>>>>> ______________________________****_________________ >>>>>>>>> >>>>>>>> rsyslog mailing list >>>>>>>> http://lists.adiscon.net/****mailman/listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>>>>>>> <http:**//lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>>>>>>> > >>>>>>>> http://www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>>>>>> <http://**www.rsyslog.com/professional-**services/<http://www.rsyslog.com/professional-services/> >>>>>>>> > >>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT >>>>>>>> POST if >>>>>>>> you DON'T LIKE THAT. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>> ______________________________****_________________ >>>>> >>>> rsyslog mailing list >>>> http://lists.adiscon.net/****mailman/listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>>> <http:**//lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>>> > >>>> http://www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>> <http://**www.rsyslog.com/professional-**services/<http://www.rsyslog.com/professional-services/> >>>> > >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>> DON'T LIKE THAT. >>>> >>>> ______________________________****_________________ >>>> >>> rsyslog mailing list >>> http://lists.adiscon.net/****mailman/listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>> <http:**//lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>> > >>> http://www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>> <http://**www.rsyslog.com/professional-**services/<http://www.rsyslog.com/professional-services/> >>> > >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>> DON'T LIKE THAT. >>> >>> ______________________________**_________________ >> rsyslog mailing list >> http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >> http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/> >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> >> ______________________________**_________________ > rsyslog mailing list > http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> > http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/> > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
