Hello list,
In our company we use puppet to configure our ec2 instances.
we start our servers with fresh instances debian lenny, some small and
some large.
we install puppet like this:
apt-get -y --force-yes install ruby1.8 rubygems1.8
gem install puppet --include-dependencies --version 0.24.8 --no-rdoc --
no-ri
ln -s /var/lib/gems/1.8/bin/puppetd /usr/sbin/puppetd
ln -s /var/lib/gems/1.8/bin/puppet /usr/sbin/puppet
ln -s /var/lib/gems/1.8/bin/facter /usr/bin/facter
and create a puppet client config file:
echo "[puppetd]" > /etc/puppet/puppetd.conf
echo "logdir=/var/log/puppet" >> /etc/puppet/puppetd.conf
echo "vardir=/var/lib/puppet" >> /etc/puppet/puppetd.conf
echo "rundir=/var/run" >> /etc/puppet/puppetd.conf
echo "server=<%=env_puppetmaster_fqdn%>" >> /etc/puppet/puppetd.conf
echo "masterport=<%=pm_apache2_lb_port%>" >> /etc/puppet/puppetd.conf
echo "pluginsync=true" >> /etc/puppet/puppetd.conf
echo "factpath = /var/lib/puppet/lib/facter" >> /etc/puppet/
puppetd.conf
and start a first puppetrun with
/usr/sbin/puppetd --test -v --waitforcert 120
The problem:
sometimes (not always) the client "breaks", uses 100%CPU, runs
manifest very very slowly and sometimes never finishes.
Killing the Client after
puppetd -v --test --debug shows:
debug: //Node[default]/milter-sms/milter-sms::mimedefang/Runit_service
[mimedefang-multiplexor]/File[run-logfile mimedefang-multiplexor]/
checksum: Initializing checksum hash
debug: //Node[default]/milter-sms/milter-sms::mimedefang/Runit_service
[mimedefang-multiplexor]/File[run-logfile mimedefang-multiplexor]:
Creating checksum {md5}d6a4b36fea3bf1f7cd0c6e70989ca8c7
notice: Caught INT; shutting down
debug: Signal caught here:
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `call'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
262:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `each'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
262:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `each'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
262:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `each'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
262:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `each'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
262:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `each'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
262:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `each'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
262:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `each'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
262:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `each'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
262:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `each'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
262:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `each'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
262:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `each'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
262:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `each'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
262:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `each'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
262:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `each'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
262:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `each'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb:
260:in `walk'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/pgraph.rb:
116:in `tree_from_vertex'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/pgraph.rb:37:in
`dependencies'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb:
346:in `failed_dependencies?'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb:
619:in `skip?'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb:
219:in `eval_resource'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb:
310:in `evaluate'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/util.rb:425:in
`thinmark'
debug: /usr/lib/ruby/1.8/benchmark.rb:308:in `realtime'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/util.rb:424:in
`thinmark'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb:
309:in `evaluate'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb:
303:in `collect'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb:
303:in `evaluate'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/node/catalog.rb:
124:in `apply'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/network/client/
master.rb:256:in `run'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/util.rb:180:in
`benchmark'
debug: /usr/lib/ruby/1.8/benchmark.rb:308:in `realtime'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/util.rb:179:in
`benchmark'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/network/client/
master.rb:255:in `run'
debug: /usr/lib/ruby/1.8/sync.rb:230:in `synchronize'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/network/client/
master.rb:237:in `run'
debug: /var/lib/gems/1.8/gems/puppet-0.24.8/bin/puppetd:417
debug: /usr/sbin/puppetd:19:in `load'
debug: /usr/sbin/puppetd:19
notice: Shutting down
debug: Storing state
debug: Stored state in 0.37 seconds
debug: Storing state
debug: Stored state in 0.52 seconds
The problem:
sometimes this never happens on one ec2-instance.
we can start new "fresh" instances and sometimes it always works with
the same manifests file.
we have different "roles" in our environment with different manifests/
classes/modules and when we start a puppetrun on role1 and another on
role2 it happens that role2 fails and role1 not. Starting the run
again sometimes both fail, sometimes both are ok, sometimes they
switch.
puppet mostly "stops" when creating checksums of configuration files
(template based or via fileserver-mount) and at the end of the
puppetrun.
Files which are checked at the beginning of the puppetrun (because
this Files are first checked because of require/subscribe direcives)
always gets done fast. Files which are checked at the end hangs on
md5summing.
The thing is that I cant reproduce this problem when I want to.
Deleting /var/lib/puppet/* and running the client does not always
helps.
Anyone experiences the same problem sometimes or could give a hint
what it could be?
Many thanks,
Kurt B.
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---
