Issue #9854 has been updated by Michael Arlt.
Assignee set to Ken Barber
> $extlookup_datadir = $myclass::extlookup_datadir
> extlookup("foobar")
>
> As far as fixing extlookup to support local variables, I don't myself see a
> problem - but I'm asking around to find out the situation. I'm sure Markus
> had a reason for changing this, and I need to determine if there is any risk
> in reverting it.
In practice you want to define $extlookup_datadir global and
$extlookup_precendence in your class.
It is reasonable to decide that dynamic lookup isn't supported anymore - but
then local used variables is a must.
It makes no sense to look in the global namespace for extlookup because you
can't influence the data retrieval in your class.
Your example above must work but won't without this naive change to
extlookup.rb. As a user i see a normal usage in the following way:
$extlookup_datadir = "${::extlookup_datadir}"
$extlookup_precedence = ['myclass','global']
extlookup("foobar")
This must work.
----------------------------------------
Bug #9854: extlookup_datadir and extlookup_precedence not being set in puppet
2.7.1
https://projects.puppetlabs.com/issues/9854#change-66971
Author: Lex van Roon
Status: Needs Decision
Priority: Urgent
Assignee: Ken Barber
Category: server
Target version: 2.7.x
Affected Puppet version: 2.7.1
Keywords:
Branch:
During tests on a puppet 2.7.1 setup, we've discovered that we could no longer
use extlookups. With 2.6, we use the following (cleaned up) manifest to perform
extlookups. Note that we use a custom fact to derive the zone + location
variables from the domain name, which are 'core' and 'ams01' in our situation.
We derive the datadir from the currently used module using an inline template.
class test {
$extlookup_datadir = inline_template("<%=
Puppet::Node::Environment.current.module('${module_name}').path %>/extdata")
$extlookup_precedence = ["hosts/%{fqdn}", "%{zone}-%{location}",
"%{location}", inline_template("<%= if defined? @location then
location.gsub(/[^a-z\s]/, \"\") end%>"), 'default']
notify { "extlookup_datadir: ${extlookup_datadir}":; }
notify { "extlookup_precedence: ${extlookup_precedence}":; }
$msg = extlookup("hello")
notify { "hello ${msg}":; }
notify { "zone: ${zone}":; }
notify { "location: ${location}":; }
}
For the purpose of this bugreport, I've got one variable defined in my CSV file
named 'hello' with the value of 'world'. Using puppet 2.7.1, this gives us the
following error:
root@puppetmaster002:~# puppet agent --test
info: Retrieving plugin
info: Loading facts in environment
info: Loading facts in interfaces
info: Loading facts in zone
info: Loading facts in puppet_var_path
info: Loading facts in acpi_available
info: Loading facts in mountpoints
info: Loading facts in location
info: Loading facts in environment
info: Loading facts in interfaces
info: Loading facts in zone
info: Loading facts in puppet_var_path
info: Loading facts in acpi_available
info: Loading facts in mountpoints
info: Loading facts in location
err: Could not retrieve catalog from remote server: Error 400 on SERVER: No
match found for 'hello' in any data file during extlookup()
at /etc/puppet/modules/shared/test/manifests/init.pp:9 on node
puppetmaster002.core.ams01.domain.tld
warning: Not using cache on failed catalog
err: Could not retrieve catalog; skipping run
Both this output and the puppetmaster logs do not give any hints as to whats
going on (see #7158), nor do debug/verbose mode give more info. Using strace
attached to the puppetmaster process, I found that the puppetmaster process is
trying to open the following CSV file:
root@puppetmaster002:~# grep stat\(.*csv /tmp/out
14959 stat("/.csv", 0x7fff144f5390) = -1 ENOENT (No such file or
directory)
This leads me to think that both extlookup_datadir and extlookup_preference are
not being parsed correctly under 2.7.1, or are being ignored completely. Using
puppet 2.6.8, we get the following output:
root@puppetmaster002:~# puppet agent --test
info: Retrieving plugin
info: Loading facts in environment
info: Loading facts in acpi_available
info: Loading facts in puppet_var_path
info: Loading facts in location
info: Loading facts in mountpoints
info: Loading facts in zone
info: Loading facts in interfaces
info: Loading facts in environment
info: Loading facts in acpi_available
info: Loading facts in puppet_var_path
info: Loading facts in location
info: Loading facts in mountpoints
info: Loading facts in zone
info: Loading facts in interfaces
info: Caching catalog for puppetmaster002.core.ams01.sn.ecg
info: Applying configuration version '1317646685'
notice: hello world
notice: /Stage[main]/Test/Notify[hello world]/message: defined 'message' as
'hello world'
notice: extlookup_precedence:
hosts/%{fqdn}%{zone}-%{location}%{location}amsdefault
notice: /Stage[main]/Test/Notify[extlookup_precedence:
hosts/%{fqdn}%{zone}-%{location}%{location}amsdefault]/message: defined
'message'
as 'extlookup_precedence:
hosts/%{fqdn}%{zone}-%{location}%{location}amsdefault'
notice: location: ams01
notice: /Stage[main]/Test/Notify[location: ams01]/message: defined
'message' as 'location: ams01'
notice: extlookup_datadir: /etc/puppet/modules/shared/test/extdata
notice: /Stage[main]/Test/Notify[extlookup_datadir:
/etc/puppet/modules/shared/test/extdata]/message: defined 'message' as
'extlookup_datadir: /etc/puppet/modules/shared/test/extdata'
notice: zone: core
notice: /Stage[main]/Test/Notify[zone: core]/message: defined 'message' as
'zone: core'
notice: Finished catalog run in 0.05 seconds
which is correct. Verifying the opened files with strace gives us the following:
root@puppetmaster002:~# grep stat\(.*csv /tmp/out
29848 stat("/etc/puppet/modules/shared/test/extdata/.csv", 0x7fffdf3660d0)
= -1 ENOENT (No such file or directory)
29848
stat("/etc/puppet/modules/shared/test/extdata/hosts/puppetmaster002.core.ams01.domain.tld.csv",
{st_mode=S_IFREG|0644, st_size=1087, ...}) = 0
29848 stat("/etc/puppet/modules/shared/test/extdata/core-ams01.csv",
{st_mode=S_IFREG|0644, st_size=12, ...}) = 0
What's weird is that the first stat() tries to open
'/etc/puppet/modules/shared/test/extdata/.csv', which could be due to the way
we construct extlookup_precedence, but the last line does open the correct CSV
file and retrieves the correct variable.
When I compare extlookup.rb between 2.6.8 and 2.7.1, I notice the following
differences:
root@puppetmaster002:~# diff /tmp/extlookup.rb
/usr/lib/ruby/1.8/puppet/parser/functions/extlookup.rb
94,95c94
< extlookup_datadir = lookupvar('extlookup_datadir')
< extlookup_precedence = Array.new
---
> extlookup_datadir = undef_as('',lookupvar('::extlookup_datadir'))
97,101c96
< extlookup_precedence = lookupvar('extlookup_precedence').collect do
|var|
< var.gsub(/%\{(.+?)\}/) do |capture|
< lookupvar($1)
< end
< end
---
> extlookup_precedence =
undef_as([],lookupvar('::extlookup_precedence')).collect { |var|
var.gsub(/%\{(.+?)\}/) { lookupvar("::#{$1}") } }
So there have been some changes in the way extlookup_datadir and
extlookup_precedence are parsed which are probably causing this behaviour. My
Ruby skills are not good enough to understand what's going on here and to write
a patch unfortunately.
This is currently keeping us from migrating to 2.7.
--
You have received this notification because you have either subscribed to it,
or are involved in it.
To change your notification preferences, please click here:
http://projects.puppetlabs.com/my/account
--
You received this message because you are subscribed to the Google Groups
"Puppet Bugs" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/puppet-bugs?hl=en.
