Issue #20865 has been updated by Charlie Sharpsteen. Category set to transactions Status changed from Investigating to Accepted Assignee deleted (Charlie Sharpsteen) Priority changed from Normal to High
Thanks a bunch for the additional info---after investigating this problem yesterday, I agree that the cronjob probably isn't related. Here is what I think is going on: Resources are evaluated and applied by the agent in a Transaction which loops over each entry in the catalog and: - Evaluates the resource unless it is skipped due to something like failed dependencies or missing tags. - Processes any events that the resource is subscribed to. In the case of refresh events, this can trigger a second evaluation of the resource. During evaluation, the following steps occur: - The current state of the resource is retrieved. In the case of Exec resources, retrieving state can include running `onlyif` checks. - The retrieved state is compared against the desired state defined in the catalog. - Any differences are resolved in a sync step. The problem is that `refreshonly` Exec resources are a bit of an oddity: the Transaction has no concept of `refreshonly` so it is left up to the Exec to pull a bunch of internal monkey business and figure out if it is being evaluated or refreshed and take the appropriate action. Unfortunately, this monkey business comes during the sync step which happens after the `onlyif` checks are fired during retrieval. To recap: the Exec resource is retrieved by the Transaction and `onlyif` is fired before the resource has a chance to respond appropriately to `refreshonly` (the appropriate response would be to skip retrieval entirely). This brings us to the random behavior bit. During retrieval, the resource cycles through a list of the following parameters and checks them: <pre> [:refreshonly, :creates, :unless, :onlyif] </pre> The iteration will break early if any of the parameter checks indicate that the resource needs to be refreshed. The list of checks is generated by grabbing the keys from a hash---which under Ruby 1.8.7 is a non-deterministic operation. So under 1.8.7, sometimes `:refreshonly` will get checked first and the operation will stop there. Sometimes `:onlyif` will get checked first, causing the onlyif command to execute, and then the checking will stop. ---------------------------------------- Bug #20865: Random behavior when notifying exec ressources having "onlyif" parameter set https://projects.puppetlabs.com/issues/20865#change-92177 * Author: Christian Flamm * Status: Accepted * Priority: High * Assignee: * Category: transactions * Target version: * Affected Puppet version: 3.1.1 * Keywords: exec onlyif notify * Branch: ---------------------------------------- Please have a look at this simplified example. class runtimevariation { # dummy file ressources that should represent # some real software deployment file { '/tmp/software-v1': ensure => present, } file { '/tmp/software-v2': ensure => present, } file { '/tmp/software-v3': ensure => present, } # indicating current version in use file { '/tmp/current': ensure => link, target => '/tmp/software-v2', require => File['/tmp/software-v2'], notify => Exec['software.restart'], } # whenever link target changes I want to run this - but # only if the file exists and is executable exec { 'software.restart': refreshonly => true, command => '/tmp/restart/software', onlyif => '/usr/bin/[ -x /tmp/restart/software ]' } } On the configured machine: # ll /tmp/ total 0 lrwxrwxrwx 1 root root 16 May 23 08:41 current -> /tmp/software-v2 -rw-r--r-- 1 root root 0 May 23 08:40 software-v1 -rw-r--r-- 1 root root 0 May 23 08:41 software-v2 -rw-r--r-- 1 root root 0 May 23 08:41 software-v3 There are a couple of different '/tmp/software-v*' file resources. There's a '/tmp/current' link pointing to either one of them. Whenever something about the link file resource (e.g. target) changes I want to notify the 'software.restart' exec resource to run a command but only if a matching executable exists. **My assumption was**: if nothing about the whole catalog/class changes (particularly the '/tmp/current' link) then the exec resource would never get notified and so neither the onlyif check nor the command would be run. **But: the onlyif check does sometimes get run**. I just don't understand when and why. During the following puppet agent runs nothing about the node's catalog (only containing class runtimevariation) changes. # puppet agent --no-daemonize --debug --onetime | grep "/usr/bin/\[ -x " Debug: Exec[software.restart](provider=posix): Executing check '/usr/bin/[ -x /tmp/restart/software ]' Debug: Executing '/usr/bin/[ -x /tmp/restart/software ]' # puppet agent --no-daemonize --debug --onetime | grep "/usr/bin/\[ -x " nothing! # puppet agent --no-daemonize --debug --onetime | grep "/usr/bin/\[ -x " Debug: Exec[software.restart](provider=posix): Executing check '/usr/bin/[ -x /tmp/restart/software ]' Debug: Executing '/usr/bin/[ -x /tmp/restart/software ]' # puppet agent --no-daemonize --debug --onetime | grep "/usr/bin/\[ -x " nothing! # puppet agent --no-daemonize --debug --onetime | grep "/usr/bin/\[ -x " nothing! # puppet agent --no-daemonize --debug --onetime | grep "/usr/bin/\[ -x " Debug: Exec[software.restart](provider=posix): Executing check '/usr/bin/[ -x /tmp/restart/software ]' Debug: Executing '/usr/bin/[ -x /tmp/restart/software ]' # puppet agent --no-daemonize --debug --onetime | grep "/usr/bin/\[ -x " nothing! Sometimes the onlyif check does get executed... sometimes not. This becomes more interessting if you have a bunch of such exec ressources with (more) expensive onlyif checks. In our setup this makes a difference of 15-20s vs. 30-35s per catalog run. -- You have received this notification because you have either subscribed to it, or are involved in it. To change your notification preferences, please click here: http://projects.puppetlabs.com/my/account -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/puppet-bugs?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
