** Description changed: - Good afternoon. - I have started seeing something very similar to Debian Dug 734688 "Logs are not rotated for a month" but in the latest Ubuntu LTS (16.04). I seem to have + [Impact] + + Logrotate fails to rotate a log and then will continue to fail to rotate + it until manual intervention takes place. If messaging has not been + configured on the system there will be no warning issued to the user. + The log will grow day by day until a user intervenes or the drive that + the log is stored on is full. + + Very large log files can make it more difficult to find useful data. + Full drives make the rest of the system fail to operate. Backporting a + fix would prevent drives filling up on stable releases. + + [Test Case] + + Go to your logs area (/var/logs) and create a file with a name ending + .2, as would be created part way through the logrotate process. So if + you have /var/log/syslog, /var/log/syslog.1, /var/log/syslog.2.gz, + /var/log/syslog.3.gz; create a new file named /var/log/syslog.2. Your + subsequent log rotate runs will fail. + + [Regression Potential] + + - I'd hope the potential is low as it only triggers under certain conditions that are special (target filename in the way). + - So far in those conditions it failed to rotate + - Yet If despite my hope there still manifests an issue I'd expect it could be renaming files it should not, so people would end up "missing" their logs - the good thing is that this is a rename, so they should find it at different names. + - Another thing I consider possible is that some unexpected conditions cause e.g. a crash in the changed code, in that case the logs are not rotated, but since there is no unlink the logs will still exist. + - Therefore I consider the Potential low enough to consider the fix. + + [Other Info] + n/a + + --- + + Good afternoon. + I have started seeing something very similar to Debian Dug 734688 "Logs are not rotated for a month" but in the latest Ubuntu LTS (16.04). I seem to have $ logrotate --version logrotate 3.8.7 bundled in it. A few weeks ago I started getting root emails such as this: > Subject: Cron <root@warden> test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily ) > > /etc/cron.daily/logrotate: > error: error creating output file /var/log/munin/munin-node.log.1: File exists > run-parts: /etc/cron.daily/logrotate exited with return code 1 - When I inspected the area of concern I was able to see that there was an existing .1 file. + When I inspected the area of concern I was able to see that there was an existing .1 file. manager@warden:/var/log/munin$ ll total 580 drwxr-xr-x 2 munin adm 4096 Sep 27 06:31 ./ drwxr-xr-x 13 root syslog 4096 Oct 5 06:26 ../ -rw-r--r-- 1 root root 3440 Sep 26 13:39 munin-node-configure.log -rw-r--r-- 1 root root 490251 Oct 5 10:25 munin-node.log -rw-r--r-- 1 root root 56598 Sep 21 02:01 munin-node.log.1 -rw-r--r-- 1 root root 24576 Aug 31 02:01 munin-node.log.2 -rw-r--r-- 1 root root 1906 Sep 19 06:25 munin-node.log.8.gz - The contents of the munin-node.log file seem to run from the 19th September until today. Unlike other parts of this bug the .1 and .2 files do not seem to be already compressed. + The contents of the munin-node.log file seem to run from the 19th September until today. Unlike other parts of this bug the .1 and .2 files do not seem to be already compressed. I deleted all but the munin-node.log file to see if it would resolve the problem and was going to leave it at that. Then I noticed that I have had another Ubuntu machine which has been sending similar emails for the past week: > Subject: Cron <root@trac> test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily ) > > /etc/cron.daily/logrotate: > error: error creating output file /var/log/syslog.1.gz: File exists > run-parts: /etc/cron.daily/logrotate exited with return code 1 - Different file, different machine but a very similar error message. - - Checking on the syslog file I can see that it better fits with other reports on this bug as my duplicated .1 files has a corresponding .1.gz file. + Different file, different machine but a very similar error message. + + Checking on the syslog file I can see that it better fits with other reports on this bug as my duplicated .1 files has a corresponding .1.gz file. manager@trac:/var/log$ ll syslog* -rw-r----- 1 syslog adm 918492 Oct 5 10:30 syslog -rw-r----- 1 syslog adm 125819 Sep 30 06:25 syslog.1 -rw-r----- 1 syslog adm 20638 Oct 2 02:01 syslog.1.gz -rw-r----- 1 syslog adm 41989 Sep 30 02:00 syslog.2.gz -rw-r----- 1 syslog adm 18654 Sep 28 02:01 syslog.3.gz -rw-r----- 1 syslog adm 31720 Sep 26 06:40 syslog.4.gz -rw-r----- 1 syslog adm 33151 Sep 25 02:01 syslog.5.gz -rw-r----- 1 syslog adm 17290 Sep 23 02:01 syslog.6.gz -rw-r----- 1 syslog adm 39275 Sep 21 06:35 syslog.7.gz There has been some speculation that a full or nearly full /var partition would cause this issue. I can confirm that /var is part of / on my systems and that presently both of them have several gigabytes of space. I run Munin an Icinga to monitor system state. Neither of these show / being completely full in the past month. They have both had /boot fill significantly. Trac had a highest use value of / being 99.28% full in the past year but warden has only had a peak of 33% in the past year. - A quick search of the internet suggests a couple of other people reporting similar issues: + A quick search of the internet suggests a couple of other people reporting similar issues: https://github.com/gitlabhq/gitlabhq/issues/6894 - http://raspberrypi.stackexchange.com/questions/22545/why-are-system-logs-not-rotating + http://raspberrypi.stackexchange.com/questions/22545/why-are-system-logs-not-rotating I do not believe that I have altered by logrotate configuration files but here is a copy of the ones I know about: - $ cat /etc/logrotate.conf + $ cat /etc/logrotate.conf # see "man logrotate" for details # rotate log files weekly weekly # use the syslog group by default, since this is the owning group # of /var/log/syslog. su root syslog # keep 4 weeks worth of backlogs rotate 4 # create new (empty) log files after rotating old ones create # uncomment this if you want your log files compressed #compress # packages drop log rotation information into this directory include /etc/logrotate.d # no packages own wtmp, or btmp -- we'll rotate them here /var/log/wtmp { - missingok - monthly - create 0664 root utmp - rotate 1 + missingok + monthly + create 0664 root utmp + rotate 1 } /var/log/btmp { - missingok - monthly - create 0660 root utmp - rotate 1 + missingok + monthly + create 0660 root utmp + rotate 1 } # system-specific logs may be configured here - $ cat /etc/logrotate.d/rsyslog + $ cat /etc/logrotate.d/rsyslog /var/log/syslog { - rotate 7 - daily - missingok - notifempty - delaycompress - compress - postrotate - reload rsyslog >/dev/null 2>&1 || true - endscript + rotate 7 + daily + missingok + notifempty + delaycompress + compress + postrotate + reload rsyslog >/dev/null 2>&1 || true + endscript } /var/log/mail.info /var/log/mail.warn /var/log/mail.err /var/log/mail.log /var/log/daemon.log /var/log/kern.log /var/log/auth.log /var/log/user.log /var/log/lpr.log /var/log/cron.log /var/log/debug /var/log/messages { - rotate 4 - weekly - missingok - notifempty - compress - delaycompress - sharedscripts - postrotate - reload rsyslog >/dev/null 2>&1 || true - endscript - } - - manager@trac:/var/log$ cat /etc/logrotate.d/munin-node + rotate 4 + weekly + missingok + notifempty + compress + delaycompress + sharedscripts + postrotate + reload rsyslog >/dev/null 2>&1 || true + endscript + } + + manager@trac:/var/log$ cat /etc/logrotate.d/munin-node /var/log/munin/munin-node.log { - daily - missingok - rotate 7 - compress - copytruncate - notifempty - create 640 root root + daily + missingok + rotate 7 + compress + copytruncate + notifempty + create 640 root root } I am confident that is all the files which are relative to the configuration: manager@trac:/var/log$ rgrep syslog /etc/logrotate.* /etc/logrotate.conf:# use the syslog group by default, since this is the owning group /etc/logrotate.conf:# of /var/log/syslog. /etc/logrotate.conf:su root syslog /etc/logrotate.d/ufw: invoke-rc.d rsyslog reload >/dev/null 2>&1 || true /etc/logrotate.d/rsyslog:/var/log/syslog /etc/logrotate.d/rsyslog: reload rsyslog >/dev/null 2>&1 || true /etc/logrotate.d/rsyslog: reload rsyslog >/dev/null 2>&1 || true manager@trac:/var/log$ rgrep munin /etc/logrotate.* /etc/logrotate.d/munin-node:/var/log/munin/munin-node.log If there is anything else I can do to help diagnose the issue please let me know. - Usual boiler plate bits: + Usual boiler plate bits: "trac" machine: $ lsb_release -rd Description: Ubuntu 14.04.5 LTS Release: 14.04 $ apt-cache policy logrotate logrotate: - Installed: 3.8.7-1ubuntu1 - Candidate: 3.8.7-1ubuntu1 - Version table: - *** 3.8.7-1ubuntu1 0 - 500 http://gb.archive.ubuntu.com/ubuntu/ trusty/main amd64 Packages - 100 /var/lib/dpkg/status + Installed: 3.8.7-1ubuntu1 + Candidate: 3.8.7-1ubuntu1 + Version table: + *** 3.8.7-1ubuntu1 0 + 500 http://gb.archive.ubuntu.com/ubuntu/ trusty/main amd64 Packages + 100 /var/lib/dpkg/status "warden" machine: $ lsb_release -rd Description: Ubuntu 16.04.1 LTS Release: 16.04 - $ apt-cache policy logrotate + $ apt-cache policy logrotate logrotate: - Installed: 3.8.7-2ubuntu2 - Candidate: 3.8.7-2ubuntu2 - Version table: - *** 3.8.7-2ubuntu2 500 - 500 http://gb.archive.ubuntu.com/ubuntu xenial/main amd64 Packages - 100 /var/lib/dpkg/status + Installed: 3.8.7-2ubuntu2 + Candidate: 3.8.7-2ubuntu2 + Version table: + *** 3.8.7-2ubuntu2 500 + 500 http://gb.archive.ubuntu.com/ubuntu xenial/main amd64 Packages + 100 /var/lib/dpkg/status N: Ignoring file '50unattended-upgrades.ucf-dist' in directory '/etc/apt/apt.conf.d/' as it has an invalid filename extension ProblemType: Bug DistroRelease: Ubuntu 14.04 Package: logrotate 3.8.7-1ubuntu1 ProcVersionSignature: Ubuntu 4.2.0-42.49~14.04.1-generic 4.2.8-ckt12 Uname: Linux 4.2.0-42-generic x86_64 ApportVersion: 2.14.1-0ubuntu3.21 Architecture: amd64 Date: Wed Oct 5 11:14:53 2016 InstallationDate: Installed on 2015-12-14 (295 days ago) InstallationMedia: Ubuntu-Server 14.04.3 LTS "Trusty Tahr" - Beta amd64 (20150805) SourcePackage: logrotate UpgradeStatus: No upgrade log present (probably fresh install)
-- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1630516 Title: Logrotate doesn't clean old system logs, allowing them to fill the disk To manage notifications about this bug go to: https://bugs.launchpad.net/hundredpapercuts/+bug/1630516/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
