Issue #21577 has been updated by Andrew Parker.
Alright now for a bit of an update as to what is going wrong. Puppet has a way of loading facts that will pull from the `factpath` as well as the `modulepath`. This method is used when the puppet agent or puppet apply try to get facts for compiling a catalog. This method also bypasses Facter's normal mechanism for loading facts. You can find it at <https://github.com/puppetlabs/puppet/blob/53241a91c687336a273291aa6e71f90fc7752581/lib/puppet/indirector/facts/facter.rb#L27-L45>. Since facter doesn't actually know about the factpath, which is part of the libdir, which is where the plugindest is, it can't resolve dependencies itself for the other pluginsynced facts. So in this case, when the `tu_affinity_tier.rb` file got loaded by this mechanism, *and the `tu_gateway_default.rb` file had not yet*, then the `Facter.value(:tu_gateway_default)` results in `nil` because Facter doesn't yet know about that fact and doesn't know where to load it. The load order is important and explains why this happened on some systems but not others. There are several places in here where the code doesn't match up. * factpath is a character separated list of directories in which to find facts, but is passed directly to `Facter.search(*paths)` without converting it into an `Array` of those paths. Facter doesn't know how to deal with that character separated version. * factpath is assumed to be in the plugindest, but there isn't actually anything to guarantee that * the indirector code uses facts from the modulepath even when running on the agent. This means that agent can end up with facts that are not part of the pluginsync * Facter will sometimes fall back to loading everything that it can find, if there isn't any shortcut to getting a fact by loading a file based on the fact's name (`foo.rb` for a fact called `foo`). However, it will only do this once and it never resets that even if `Facter.clear` is called. So even if facter did have the right list of directories, could still sometimes miss loading the facts. * Puppet performs a `Dir.chdir` to the directory that contains the fact ruby file to load before loading it. This is very different from what Facter does. If facts are well written (don't depend on what directory it is executing in, contain all logic inside the `setcode` block), then none of these matter. However there are many cases that these inconsistencies can cause problems, as has been illustrated in this issue. ---------------------------------------- Bug #21577: Facter appears to "lose" the FACTERLIB / factpath path under certain conditions on AIX. https://projects.puppetlabs.com/issues/21577#change-94789 * Author: Zachary Stern * Status: Accepted * Priority: Normal * Assignee: * Category: * Target version: * Keywords: customer * Branch: * Affected Facter version: 1.6.17 ---------------------------------------- It appears that under a certain scenario, when using custom facts, facter on AIX "loses" FACTERLIB / factpath during puppet runs. The setting `factpath` is somehow not being honored. I'm unable to figure out what the exact cause is, but I am able to provide steps to reproduce the error. The two attached custom facts exhibit this issue, tested with PE 2.8.1 on AIX 6.1. When these facts are synced to the AIX agent via pluginsync, the following is output during a run: info: Loading facts in /opt/freeware/var/lib/pe-puppet/lib/facter/tu_affinity_tier.rb warning: Could not load fact file /opt/freeware/var/lib/pe-puppet/lib/facter/tu_affinity_tier.rb: private method `split' called for nil:NilClass info: Loading facts in /opt/freeware/var/lib/pe-puppet/lib/facter/tu_gateway_default.rb The facts in tu_affinity_tier.rb depend on the tu_gateway_default fact. When this fact (tu_gateway_default) is loaded inside of the tu_affinity_tier.rb file during a puppet run, as such - `gateway = Facter.value(:tu_gateway_default)` - it returns nil, and gateway is set to nil. However, the tu_gateway_default fact definitely works, and does not return nil. `facter -p tu_gateway_default` returns the default gateway of tha AIX system, as expected. In addition, the two facts in tu_affinity_tier.rb also seem to work outside of puppet. E.g. if I copy tu_affinity_tier.rb to my home directory and modify it with a require 'facter' at the beginning, and run it in pry, I receive the following: bash-3.2# export FACTERLIB=`puppet agent --configprint factpath | awk '{ print $1 }'` bash-3.2# echo $FACTERLIB /opt/freeware/var/lib/pe-puppet/lib/facter:/opt/freeware/var/lib/pe-puppet/facts bash-3.2# /opt/puppet/bin/ruby tu_affinity_tier.rb bash-3.2# As you can see, no error about calling split on nil is thrown. However, if I remove facterlib, e.g. `unset FACTERLIB` we do get the split on nil error, since of course facter doesn't know where the custom facts are when we use it in this way. E.g.: bash-3.2# unset FACTERLIB bash-3.2# echo $FACTERLIB bash-3.2# /opt/puppet/bin/ruby tu_affinity_tier.rb tu_affinity_tier.rb:5: private method `split' called for nil:NilClass (NoMethodError) To confirm this hypothesis, I set the code in tu_affinity_tier to use builtin fact like ipaddress, instead of the custom fact. This way, FACTERLIB / factpath doesn't come into play. It then works perfectly fine. E.g. if we change `gateway = Facter.value(:tu_gateway_default)` to `gateway = Facter.value(:ipaddress)`, and do a puppet run, the error goes away. All this despite the fact that the tu_gateway_default fact definitely works, and does not, on its own, return nil. E.g.: bash-3.2# facter -p tu_gateway_default 10.16.77.1 In addition, the facts themselves work fine outside of a puppet run, when we call them directly, e.g.: bash-3.2# export FACTERLIB=`puppet agent --configprint factpath | awk '{ print $1 }'` bash-3.2# /opt/puppet/bin/pry [1] pry(main)> require 'facter' => true [2] pry(main)> Facter.value(:tu_affinity) => "not_defined" (yeah, I'm using pry on AIX; what of it?) Note that "not_defined" is a valid resuilt for that fact. So, the code in tu_affinity_tier.rb is evaluated and no error is thrown. It ONLY happens when the code is evaluated within the scope of a puppet run. E.g.: info: Loading facts in /opt/freeware/var/lib/pe-puppet/lib/facter/tu_affinity_tier.rb warning: Could not load fact file /opt/freeware/var/lib/pe-puppet/lib/facter/tu_affinity_tier.rb: private method `split' called for nil:NilClass info: Loading facts in /opt/freeware/var/lib/pe-puppet/lib/facter/tu_gateway_default.rb HOWEVER, if I do a puppet run after explictly setting facterlib as a shell variable, there's no problem: bash-3.2# export FACTERLIB=`puppet agent --configprint factpath | awk '{ print $1 }'` bash-3.2# puppet agent -t info: Retrieving plugin info: Loading facts in /opt/freeware/var/lib/pe-puppet/lib/facter/concat_basedir.rb info: Loading facts in /opt/freeware/var/lib/pe-puppet/lib/facter/custom_auth_conf.rb info: Loading facts in /opt/freeware/var/lib/pe-puppet/lib/facter/facter_dot_d.rb info: Loading facts in /opt/freeware/var/lib/pe-puppet/lib/facter/pe_version.rb info: Loading facts in /opt/freeware/var/lib/pe-puppet/lib/facter/puppet_vardir.rb info: Loading facts in /opt/freeware/var/lib/pe-puppet/lib/facter/root_home.rb info: Loading facts in /opt/freeware/var/lib/pe-puppet/lib/facter/tu_affinity_tier.rb info: Loading facts in /opt/freeware/var/lib/pe-puppet/lib/facter/tu_gateway_default.rb info: Caching catalog for pe-aix-61-support info: Applying configuration version '1372791873' notice: Finished catalog run in 2.82 seconds Ta-da! So it looks like puppet on AIX is somehow "forgetting" the factpath parameter, under some specific scenario. -- You have received this notification because you have either subscribed to it, or are involved in it. To change your notification preferences, please click here: http://projects.puppetlabs.com/my/account -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/groups/opt_out.
