Issue #9854 has been updated by Justin Honold.
Just got confused / bit by this one myself. Definitely want to be able to set $extlookup_precedence in a class/define, as opposed to site.pp. ---------------------------------------- Bug #9854: extlookup_datadir and extlookup_precedence not being set in puppet 2.7.1 https://projects.puppetlabs.com/issues/9854#change-69071 Author: Lex van Roon Status: Needs Decision Priority: Normal Assignee: Ken Barber Category: server Target version: 2.7.x Affected Puppet version: 2.7.1 Keywords: Branch: During tests on a puppet 2.7.1 setup, we've discovered that we could no longer use extlookups. With 2.6, we use the following (cleaned up) manifest to perform extlookups. Note that we use a custom fact to derive the zone + location variables from the domain name, which are 'core' and 'ams01' in our situation. We derive the datadir from the currently used module using an inline template. class test { $extlookup_datadir = inline_template("<%= Puppet::Node::Environment.current.module('${module_name}').path %>/extdata") $extlookup_precedence = ["hosts/%{fqdn}", "%{zone}-%{location}", "%{location}", inline_template("<%= if defined? @location then location.gsub(/[^a-z\s]/, \"\") end%>"), 'default'] notify { "extlookup_datadir: ${extlookup_datadir}":; } notify { "extlookup_precedence: ${extlookup_precedence}":; } $msg = extlookup("hello") notify { "hello ${msg}":; } notify { "zone: ${zone}":; } notify { "location: ${location}":; } } For the purpose of this bugreport, I've got one variable defined in my CSV file named 'hello' with the value of 'world'. Using puppet 2.7.1, this gives us the following error: root@puppetmaster002:~# puppet agent --test info: Retrieving plugin info: Loading facts in environment info: Loading facts in interfaces info: Loading facts in zone info: Loading facts in puppet_var_path info: Loading facts in acpi_available info: Loading facts in mountpoints info: Loading facts in location info: Loading facts in environment info: Loading facts in interfaces info: Loading facts in zone info: Loading facts in puppet_var_path info: Loading facts in acpi_available info: Loading facts in mountpoints info: Loading facts in location err: Could not retrieve catalog from remote server: Error 400 on SERVER: No match found for 'hello' in any data file during extlookup() at /etc/puppet/modules/shared/test/manifests/init.pp:9 on node puppetmaster002.core.ams01.domain.tld warning: Not using cache on failed catalog err: Could not retrieve catalog; skipping run Both this output and the puppetmaster logs do not give any hints as to whats going on (see #7158), nor do debug/verbose mode give more info. Using strace attached to the puppetmaster process, I found that the puppetmaster process is trying to open the following CSV file: root@puppetmaster002:~# grep stat\(.*csv /tmp/out 14959 stat("/.csv", 0x7fff144f5390) = -1 ENOENT (No such file or directory) This leads me to think that both extlookup_datadir and extlookup_preference are not being parsed correctly under 2.7.1, or are being ignored completely. Using puppet 2.6.8, we get the following output: root@puppetmaster002:~# puppet agent --test info: Retrieving plugin info: Loading facts in environment info: Loading facts in acpi_available info: Loading facts in puppet_var_path info: Loading facts in location info: Loading facts in mountpoints info: Loading facts in zone info: Loading facts in interfaces info: Loading facts in environment info: Loading facts in acpi_available info: Loading facts in puppet_var_path info: Loading facts in location info: Loading facts in mountpoints info: Loading facts in zone info: Loading facts in interfaces info: Caching catalog for puppetmaster002.core.ams01.sn.ecg info: Applying configuration version '1317646685' notice: hello world notice: /Stage[main]/Test/Notify[hello world]/message: defined 'message' as 'hello world' notice: extlookup_precedence: hosts/%{fqdn}%{zone}-%{location}%{location}amsdefault notice: /Stage[main]/Test/Notify[extlookup_precedence: hosts/%{fqdn}%{zone}-%{location}%{location}amsdefault]/message: defined 'message' as 'extlookup_precedence: hosts/%{fqdn}%{zone}-%{location}%{location}amsdefault' notice: location: ams01 notice: /Stage[main]/Test/Notify[location: ams01]/message: defined 'message' as 'location: ams01' notice: extlookup_datadir: /etc/puppet/modules/shared/test/extdata notice: /Stage[main]/Test/Notify[extlookup_datadir: /etc/puppet/modules/shared/test/extdata]/message: defined 'message' as 'extlookup_datadir: /etc/puppet/modules/shared/test/extdata' notice: zone: core notice: /Stage[main]/Test/Notify[zone: core]/message: defined 'message' as 'zone: core' notice: Finished catalog run in 0.05 seconds which is correct. Verifying the opened files with strace gives us the following: root@puppetmaster002:~# grep stat\(.*csv /tmp/out 29848 stat("/etc/puppet/modules/shared/test/extdata/.csv", 0x7fffdf3660d0) = -1 ENOENT (No such file or directory) 29848 stat("/etc/puppet/modules/shared/test/extdata/hosts/puppetmaster002.core.ams01.domain.tld.csv", {st_mode=S_IFREG|0644, st_size=1087, ...}) = 0 29848 stat("/etc/puppet/modules/shared/test/extdata/core-ams01.csv", {st_mode=S_IFREG|0644, st_size=12, ...}) = 0 What's weird is that the first stat() tries to open '/etc/puppet/modules/shared/test/extdata/.csv', which could be due to the way we construct extlookup_precedence, but the last line does open the correct CSV file and retrieves the correct variable. When I compare extlookup.rb between 2.6.8 and 2.7.1, I notice the following differences: root@puppetmaster002:~# diff /tmp/extlookup.rb /usr/lib/ruby/1.8/puppet/parser/functions/extlookup.rb 94,95c94 < extlookup_datadir = lookupvar('extlookup_datadir') < extlookup_precedence = Array.new --- > extlookup_datadir = undef_as('',lookupvar('::extlookup_datadir')) 97,101c96 < extlookup_precedence = lookupvar('extlookup_precedence').collect do |var| < var.gsub(/%\{(.+?)\}/) do |capture| < lookupvar($1) < end < end --- > extlookup_precedence = undef_as([],lookupvar('::extlookup_precedence')).collect { |var| var.gsub(/%\{(.+?)\}/) { lookupvar("::#{$1}") } } So there have been some changes in the way extlookup_datadir and extlookup_precedence are parsed which are probably causing this behaviour. My Ruby skills are not good enough to understand what's going on here and to write a patch unfortunately. This is currently keeping us from migrating to 2.7. -- You have received this notification because you have either subscribed to it, or are involved in it. To change your notification preferences, please click here: http://projects.puppetlabs.com/my/account -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-bugs?hl=en.
