Issue #18028 has been updated by Henrik Lindberg. Status changed from Unreviewed to Needs Decision
The system behavior is undefined if the root partition is filled. Not sure it is possible to do anything with any guaranteed quality. Not sure if it is worth trying to salvage this situation; marking as "decision needed" if this is a "won't/can't fix" or a problem that is of value to do something about. ---------------------------------------- Bug #18028: puppet agent gets disabled for good on full filesystems https://projects.puppetlabs.com/issues/18028#change-78495 Author: Frederic Schaer Status: Needs Decision Priority: Normal Assignee: Category: agent Target version: Affected Puppet version: 2.7.20 Keywords: full filesystem lockfile puppetdlock enable disable Branch: Hi, I did, by mistake install a small VM, and forgot to remove a cron job that fetched a few gigabytes of data and filled the wrongly configured VM filesystems... I know, lots of issues already. So , we had a big slash filesystem, and that's it: [root@node64 data]# df -h Filesystem Size Used Avail Use% Mounted on /dev/vda2 8.8G 2.9G 5.6G 34% / tmpfs 245M 0 245M 0% /dev/shm /dev/vda1 97M 49M 43M 54% /boot The issue is : when the cron ran, it filled up /, and thus, /tmp and /var. Off course, this is the initial issue, but then puppet ran and left behind its lock file, thus disabling for good the agent, no matter what happens later. Removing the lock file, restarting the agent once, and then once again shows that behaviour : [root@node64 ~]# puppet agent -o --no-daemonize -v info: Retrieving plugin info: Loading facts in /var/lib/puppet/lib/facter/iptables_version.rb info: Loading facts in /var/lib/puppet/lib/facter/ip6tables_version.rb info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb info: Loading facts in /var/lib/puppet/lib/facter/concat_basedir.rb info: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rb info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb err: Could not retrieve catalog from remote server: Could not intern from pson: Could not autoload package: Could not autoload /usr/lib/ruby/site_ruby/1.8/puppet/provider/package/rug.rb: Could not autoload rpm: cannot generate tempfile `/tmp/puppet20121207-20992-k1ceu5-9' info: Not using expired catalog for node64### from cache; expired at Thu Dec 06 19:07:30 +0100 2012 notice: Using cached catalog err: Could not retrieve catalog; skipping run err: Could not save last run local report: cannot generate tempfile `/var/lib/puppet/state/last_run_summary.yaml20121207-20992-1egoxgq-9' [root@node64 ~]# puppet agent -o --no-daemonize -v notice: Skipping run of Puppet configuration client; administratively disabled; use 'puppet agent --enable' to re-enable. [root@node64 ~]# puppet agent -o --no-daemonize -v notice: Skipping run of Puppet configuration client; administratively disabled; use 'puppet agent --enable' to re-enable. => to get the agent back up, I did: * clean up the filesystem, off course, * but also I had to rm -f /var/lib/puppet/state/puppetdlock I know this should be avoided to have such a setup and to fill up filesystems, but this can happens And unless this is by design, I think the puppet agent should not disable itself even in case of full filesystem, should it ? puppet agent version : 2.7.20 Regards -- You have received this notification because you have either subscribed to it, or are involved in it. To change your notification preferences, please click here: http://projects.puppetlabs.com/my/account -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To post to this group, send email to puppet-bugs@googlegroups.com. To unsubscribe from this group, send email to puppet-bugs+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-bugs?hl=en.
