Issue #10418 has been updated by Jo Rhett.
Here's another strace. It's the very end of the output of "strace puppet agent
--enable". What it shows is puppet failing to remove the puppetdlock file.
There is something wrong in this piece of code:
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
stat("/var/lib/puppet/state/puppetdlock", {st_mode=S_IFREG|0644, st_size=5,
...}) = 0
open("/var/lib/puppet/state/puppetdlock", O_RDONLY) = 6
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
fstat(6, {st_mode=S_IFREG|0644, st_size=5, ...}) = 0
fstat(6, {st_mode=S_IFREG|0644, st_size=5, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x2ad97fefe000
lseek(6, 0, SEEK_CUR) = 0
read(6, "13115", 4096) = 5
read(6, "", 4096) = 0
close(6) = 0
munmap(0x2ad97fefe000, 4096) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
stat("/var/lib/puppet/state/puppetdlock", {st_mode=S_IFREG|0644, st_size=5,
...}) = 0
open("/var/lib/puppet/state/puppetdlock", O_RDONLY) = 6
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
fstat(6, {st_mode=S_IFREG|0644, st_size=5, ...}) = 0
fstat(6, {st_mode=S_IFREG|0644, st_size=5, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x2ad97fefe000
lseek(6, 0, SEEK_CUR) = 0
read(6, "13115", 4096) = 5
read(6, "", 4096) = 0
close(6) = 0
munmap(0x2ad97fefe000, 4096) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigaction(SIGINT, {SIG_DFL, [INT], SA_RESTORER|SA_RESTART, 0x301ac302d0},
{0x36b8c8ccf0, [INT], SA_RESTORER|SA_RESTART, 0x301ac302d0}, 8) = 0
close(4) = 0
munmap(0x2ad97f1a6000, 4096) = 0
close(3) = 0
munmap(0x2ad97f1a5000, 4096) = 0
exit_group(0) = ?
$ ls /var/lib/puppet/state/
graphs last_run_report.yaml last_run_summary.yaml puppetdlock state.yaml
$ cat /var/lib/puppet/state/puppetdlock
13115
(same as it was before -- the running daemon's pid)
----------------------------------------
Bug #10418: "Caught TERM; calling stop" with state/puppetdlock left in place
https://projects.puppetlabs.com/issues/10418
Author: Jo Rhett
Status: Investigating
Priority: Normal
Assignee:
Category: agent
Target version:
Affected Puppet version: 2.6.12
Keywords: enabledisable
Branch:
Mon Oct 31 23:03:31 +0000 2011 Puppet (notice): Caught TERM; calling stop
Ever since the 2.6.12 upgrade I've been seeing these reports reach us. As in,
about a hundred of a half thou machines. Most of the time we find that
$vardir/state/puppetdlock is in place and blocking further puppet runs, which
requires a manual resolution.
I wrote a quick cron script to look for puppetdlock files older than one hour,
remove them and mail me a report and I've received several dozen in the last
few hours. Something is clearly broken in 2.6.12, we are backgrading our
systems to 2.6.11.
No-- I have no other information than that it crosses all of our machine types,
and we have had no significant changes in our modules in this time period.
Many of the machines which have failed have had zero module or manifest changes
which would apply to them. I cannot get this to replicate on the command line.
--
You have received this notification because you have either subscribed to it,
or are involved in it.
To change your notification preferences, please click here:
http://projects.puppetlabs.com/my/account
--
You received this message because you are subscribed to the Google Groups
"Puppet Bugs" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/puppet-bugs?hl=en.
