Issue #12741 has been updated by Jason Antman.
I'm the guy from Chris' community ticket (Issue #2888).
I gave xray a try, but AFAIK (according to my understanding of its docs), the
thread dump won't work at all on MRI without a patch - however Ruby EE
supposedly includes this natively. I patched ruby-1.8.7.352 from CentOS as
required, installed the xray gem, and made the following change to 2.6.14's
/usr/sbin/puppetd:
#!/usr/bin/ruby
require 'rubygems'
require 'xray/thread_dump_signal_handler'
require 'puppet/application/agent'
Puppet::Application[:agent].run
The other issue that's brought up here is xray writes the thread dump to STDERR
(https://github.com/ph7/xray/blob/master/lib/xray/thread_dump_signal_handler.rb)
so, this will only work on puppetd running --no-daemonize from the console,
which is impractical to debug long-running issues, and likely introduces some
alternate code paths. I'm not enough of a Ruby programmer to know the
implications of this, but if Puppet implements it, perhaps the xray thread_dump
library can be patched to open a file handle and write the thread dumps to
(ex.) /var/lib/puppet/thread.dump instead of stderr?
----------------------------------------
Feature #12741: Add a hook for dumping thread state / backtrace of a running
puppet process
https://projects.puppetlabs.com/issues/12741#change-59683
Author: Chris Price
Status: Accepted
Priority: Normal
Assignee:
Category:
Target version:
Affected Puppet version:
Keywords:
Branch:
I was working on a ticket from the community on Friday, and came across a
situation where it would have been really valuable to be able to get a
backtrace / thread dump from a puppet agent process that he had running in his
environment.
There are a few existing tools out there that provide some of this sort of
functionality... in particular, a gem called "xray". Among other things, it
contains a file that you can require:
require 'xray/thread_dump_signal_handler'
which basically just calls Kernel.trap to add a signal handler for the "QUIT"
signal. The signal handler simply calls "caller" and prints it to stdout.
It looks like a very low-overhead snippet of code. And, actually, it's only
about 4 lines... so we could just add something like this to puppet directly
and then we wouldn't need to introduce a dependency on the gem.
Considerations:
1. is "QUIT" the best signal to trap for this?
2. we should try to have it dump all threads, not just the current one. it
seems like (for ruby 1.8.7) you can't accomplish this without patching the ruby
interpreter itself... but for 1.9 You can use "Thread#backtrace". So this code
should probably be written to try to leverage that.
--
You have received this notification because you have either subscribed to it,
or are involved in it.
To change your notification preferences, please click here:
http://projects.puppetlabs.com/my/account
--
You received this message because you are subscribed to the Google Groups
"Puppet Bugs" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/puppet-bugs?hl=en.
