Issue #21203 has been updated by Mark Jeromin. Status changed from Unreviewed to Needs Decision
Replicated problem in Puppet 2.7.23 with 100 x user & ssh_authorized_key resources, and observed successful puppet run with 36 leftover /tmp/puppet* files. Referring to product owner for decision. ---------------------------------------- Bug #21203: ssh_authorized_key causes failure of /tmp/puppet* removal if many keys are added to many users (non-deterministic) AS the :user in question, which leads to failed runs. https://projects.puppetlabs.com/issues/21203#change-98458 * Author: Kristian Kostecky * Status: Needs Decision * Priority: Normal * Assignee: * Category: provider * Target version: * Affected Puppet version: 2.7.20 * Keywords: ssh_authorized_key puppet tempfile provider * Branch: ---------------------------------------- If you create a manifest that adds many keys to many different users you will get the occasional failure depending on the order operations are run. The more resources you create, the higher chance you will hit this issue. The standard behaviour is that all tempfile.rb operations that unlink/cleanup are done as root. However, occasionally some order of operations doesn't occur due to a ssh_authorized_key causing a drop of privileges to write out a key for a resource and then a flush happens on a *different* resource (hence different tempfile associated with a previous ssh_authorized_key resource) and the unlink() is attempted as a non-root user. Hence, failing. You'll see this issue manifest in 2 ways and *not* on every run. 1 - most frequent) Puppet run will succeed, however, you will notice some /tmp/puppet-* tempfiles that ssh_authorized_key triggered the creation of that were not removed due to the fact that the code in /usr/lib/ruby/*/tempfile.rb within the callback method is running the unlink as a previously used user of one of the ssh_authorized_key resources. 2 - less frequent) Puppet run will fail. The reason is identical, it's just triggered higher up in the stack where the exception raised actually causes the puppet run to fail due to many resources failing. This happens because the "cleanlist" in the tempfile class has one operation fail, the entry is not removed and then on subsequent flushes the entry is retried on the unlink(). It continues to fail right through as this operation is not performed as root. ## WORKAROUND: * Use the ssh_authorized_key resource and set :user => 'root', :target => '<alternate location>', :notify => '<exec resource to chown and move file in to place>' * Create <exec resource to chown and move file in to place> and set :refreshonly => 'true' * This obviously works as all operations are performed as root with respect to the tempfiles created by the parsedfile class, etc... ## What I've done: * Run puppet agent with "ruby -d puppet agent ..." to enable $DEBUG = 'true' * Added some logging of UIDs, etc. throughout the code ### Relevant files: * `/usr/lib/ruby/1.8/tempfile.rb` * `/usr/lib/ruby/site_ruby/1.8/puppet/provider/ssh_authorized_key/parsed.rb` * `/usr/lib/ruby/site_ruby/1.8/puppet/util/filetype.rb` Relevant traceback: <pre> removing /tmp/puppet20130610-18363-10czkrv-0... as UID: 531 Exception `Errno::EPERM' at /usr/lib/ruby/1.8/tempfile.rb:173 - Operation not permitted - /tmp/puppet20130610-18363-10czkrv-0 /usr/lib/ruby/1.8/tempfile.rb:173:in `unlink' /usr/lib/ruby/1.8/tempfile.rb:173:in `callback' /usr/lib/ruby/1.8/fileutils.rb:1062:in `call' /usr/lib/ruby/1.8/fileutils.rb:1062:in `read' /usr/lib/ruby/1.8/fileutils.rb:1062:in `fu_copy_stream0' /usr/lib/ruby/1.8/fileutils.rb:1270:in `copy_file' /usr/lib/ruby/1.8/fileutils.rb:1269:in `open' /usr/lib/ruby/1.8/fileutils.rb:1269:in `copy_file' /usr/lib/ruby/1.8/fileutils.rb:1268:in `open' /usr/lib/ruby/1.8/fileutils.rb:1268:in `copy_file' /usr/lib/ruby/1.8/fileutils.rb:464:in `copy_file' /usr/lib/ruby/1.8/fileutils.rb:383:in `cp' /usr/lib/ruby/1.8/fileutils.rb:1402:in `fu_each_src_dest' /usr/lib/ruby/1.8/fileutils.rb:1418:in `fu_each_src_dest0' /usr/lib/ruby/1.8/fileutils.rb:1400:in `fu_each_src_dest' /usr/lib/ruby/1.8/fileutils.rb:382:in `cp' /usr/lib/ruby/site_ruby/1.8/puppet/util/filetype.rb:110:in `write' /usr/lib/ruby/site_ruby/1.8/puppet/util/filetype.rb:56:in `real_write' /usr/lib/ruby/site_ruby/1.8/puppet/util/filetype.rb:56:in `write' /usr/lib/ruby/site_ruby/1.8/puppet/provider/parsedfile.rb:95:in `flush_target' /usr/lib/ruby/site_ruby/1.8/puppet/provider/parsedfile.rb:69:in `flush' /usr/lib/ruby/site_ruby/1.8/puppet/provider/parsedfile.rb:67:in `each' /usr/lib/ruby/site_ruby/1.8/puppet/provider/parsedfile.rb:67:in `flush' /usr/lib/ruby/site_ruby/1.8/puppet/provider/parsedfile.rb:341:in `flush' /usr/lib/ruby/site_ruby/1.8/puppet/provider/ssh_authorized_key/parsed.rb:71:in `flush' /usr/lib/ruby/site_ruby/1.8/puppet/util/suidmanager.rb:62:in `asuser' /usr/lib/ruby/site_ruby/1.8/puppet/provider/ssh_authorized_key/parsed.rb:71:in `flush' /usr/lib/ruby/site_ruby/1.8/puppet/type.rb:650:in `flush' /usr/lib/ruby/site_ruby/1.8/puppet/transaction/resource_harness.rb:139:in `evaluate' /usr/lib/ruby/site_ruby/1.8/puppet/transaction.rb:49:in `apply' /usr/lib/ruby/site_ruby/1.8/puppet/transaction.rb:84:in `eval_resource' /usr/lib/ruby/site_ruby/1.8/puppet/transaction.rb:107:in `evaluate' /usr/lib/ruby/site_ruby/1.8/puppet/util.rb:430:in `thinmark' /usr/lib/ruby/1.8/benchmark.rb:308:in `realtime' /usr/lib/ruby/site_ruby/1.8/puppet/util.rb:429:in `thinmark' /usr/lib/ruby/site_ruby/1.8/puppet/transaction.rb:107:in `evaluate' /usr/lib/ruby/site_ruby/1.8/puppet/transaction.rb:319:in `traverse' /usr/lib/ruby/site_ruby/1.8/puppet/transaction.rb:103:in `evaluate' /usr/lib/ruby/site_ruby/1.8/puppet/resource/catalog.rb:141:in `apply' /usr/lib/ruby/site_ruby/1.8/puppet/configurer.rb:150:in `run' /usr/lib/ruby/site_ruby/1.8/puppet/util.rb:194:in `benchmark' /usr/lib/ruby/1.8/benchmark.rb:308:in `realtime' /usr/lib/ruby/site_ruby/1.8/puppet/util.rb:193:in `benchmark' /usr/lib/ruby/site_ruby/1.8/puppet/configurer.rb:149:in `run' /usr/lib/ruby/site_ruby/1.8/puppet/agent.rb:39:in `run' /usr/lib/ruby/site_ruby/1.8/puppet/agent/locker.rb:21:in `lock' /usr/lib/ruby/site_ruby/1.8/puppet/agent.rb:39:in `run' /usr/lib/ruby/1.8/sync.rb:230:in `synchronize' /usr/lib/ruby/site_ruby/1.8/puppet/agent.rb:39:in `run' /usr/lib/ruby/site_ruby/1.8/puppet/agent.rb:103:in `with_client' /usr/lib/ruby/site_ruby/1.8/puppet/agent.rb:37:in `run' /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:172:in `call' /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:172:in `controlled_run' /usr/lib/ruby/site_ruby/1.8/puppet/agent.rb:35:in `run' /usr/lib/ruby/site_ruby/1.8/puppet/application/agent.rb:336:in `onetime' /usr/lib/ruby/site_ruby/1.8/puppet/application/agent.rb:310:in `run_command' /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:307:in `run' /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:411:in `hook' /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:307:in `run' /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:402:in `exit_on_fail' /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:307:in `run' /usr/lib/ruby/site_ruby/1.8/puppet/util/command_line.rb:69:in `execute' /usr/bin/puppet:4 done </pre> -- You have received this notification because you have either subscribed to it, or are involved in it. To change your notification preferences, please click here: http://projects.puppetlabs.com/my/account -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/groups/opt_out.
