Hello community, here is the log from the commit of package rubygem-hiera for openSUSE:Factory checked in at 2014-10-23 14:20:19 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/rubygem-hiera (Old) and /work/SRC/openSUSE:Factory/.rubygem-hiera.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "rubygem-hiera" Changes: -------- --- /work/SRC/openSUSE:Factory/rubygem-hiera/rubygem-hiera.changes 2013-05-16 11:32:55.000000000 +0200 +++ /work/SRC/openSUSE:Factory/.rubygem-hiera.new/rubygem-hiera.changes 2014-10-23 14:20:27.000000000 +0200 @@ -1,0 +2,34 @@ +Thu Oct 16 20:37:35 UTC 2014 - [email protected] + +- update to 1.3.4 + * CVE-2014-3248 (Arbitrary Code Execution with Required Social Engineering) + On platforms with Ruby 1.9.1 or earlier, an attacker could have + Puppet execute malicious code by convincing a privileged user to + change directories to one containing the malicious code and then + run Puppet. + * Hiera 1.3.3 + - HI-239: Backport speed improvement to 1.3.x codebase, resulting + in a substantial speed increase in lookups compared to Hiera 1.3.2. + - HI-149: Remove Fedora 18 from default build targets + - HI-236: Remove Raring (Ubuntu 13.04) from build_defaults, it is EOL + - HI-185: Add Trusty (Ubuntu 14.04) support + - HI-232: Hiera should conflict/provide/replace ruby-hiera (from Ubuntu) + * Hiera 1.3.2 + - HI-179: Add RHEL 7 support for Hiera packaging. + - HI-176: Hiera would fail to find the correct ruby binary on Debian + when an alternative version was installed. Hiera now uses /usr/bin/ruby, + which fixes the issue. + - HI-178: Acceptance tests have been added for Solaris and Windowsi + vCloud machines. + - HI-115: Hiera would show an incorrect recursive_guard warning if + the same variable was interpolated twice in a hierarchy definition, + even if the usage was not recursive. + * Hiera 1.3.1 + - HI-65: Empty YAML files can raise an exception (backported to stable as HI-71) + + A complete changelog is available from: + https://docs.puppetlabs.com/hiera/1/release_notes.html + or visiting + https://github.com/puppetlabs/hiera/compare/1.2.1...1.3.4 + +------------------------------------------------------------------- Old: ---- hiera-1.2.1.gem New: ---- hiera-1.3.4.gem ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ rubygem-hiera.spec ++++++ --- /var/tmp/diff_new_pack.c39p4R/_old 2014-10-23 14:20:28.000000000 +0200 +++ /var/tmp/diff_new_pack.c39p4R/_new 2014-10-23 14:20:28.000000000 +0200 @@ -1,7 +1,7 @@ # # spec file for package rubygem-hiera # -# Copyright (c) 2013 SUSE LINUX Products GmbH, Nuernberg, Germany. +# Copyright (c) 2014 SUSE LINUX Products GmbH, Nuernberg, Germany. # # All modifications and additions to the file contributed by third parties # remain the property of their copyright owners, unless otherwise agreed @@ -16,24 +16,24 @@ # -Name: rubygem-hiera -Version: 1.2.1 -Release: 0 %define mod_name hiera %define mod_full_name %{mod_name}-%{version} - -BuildRoot: %{_tmppath}/%{name}-%{version}-build -BuildRequires: ruby-macros >= 1 +%define mod_branch -%{version} +%define mod_weight 1 +Name: rubygem-hiera +Version: 1.3.4 +Release: 0 +Summary: Light weight hierarchical data store +License: Apache-2.0 +Group: Development/Languages/Ruby Url: https://github.com/puppetlabs/hiera Source0: http://rubygems.org/gems/%{mod_full_name}.gem Source1: hiera.1 Source2: hiera.yaml -Summary: Light weight hierarchical data store -License: Apache-2.0 -Group: Development/Languages/Ruby -%define mod_branch -%{version} -%define mod_weight 1 +BuildRequires: ruby-macros >= 1 +# FIXME: use proper Requires(pre/post/preun/...) PreReq: update-alternatives +BuildRoot: %{_tmppath}/%{name}-%{version}-build %description A pluggable data store for hierarcical data @@ -63,28 +63,28 @@ %build %install -%gem_install -f +%{gem_install} -f install -Dm0644 %{SOURCE1} %{buildroot}%{_mandir}/man1/hiera.1 -install -Dm0644 %{SOURCE2} %{buildroot}%_sysconfdir/hiera.yaml +install -Dm0644 %{SOURCE2} %{buildroot}%{_sysconfdir}/hiera.yaml mv %{buildroot}%{_bindir}/hiera{,%{mod_branch}} ln -s hiera%{mod_branch} %{buildroot}%{_bindir}/hiera mkdir -p %{buildroot}%{_docdir}/%{name} -ln -s %{_libdir}/ruby/gems/%{rb_ver}/gems/%{mod_full_name}/COPYING %buildroot/%{_docdir}/%{name}/COPYING -ln -s %{_libdir}/ruby/gems/%{rb_ver}/gems/%{mod_full_name}/README.md %buildroot/%{_docdir}/%{name}/README.md -ln -s %{_libdir}/ruby/gems/%{rb_ver}/gems/%{mod_full_name}/LICENSE %buildroot/%{_docdir}/%{name}/LICENSE +ln -s %{_libdir}/ruby/gems/%{rb_ver}/gems/%{mod_full_name}/COPYING %{buildroot}/%{_docdir}/%{name}/COPYING +ln -s %{_libdir}/ruby/gems/%{rb_ver}/gems/%{mod_full_name}/README.md %{buildroot}/%{_docdir}/%{name}/README.md +ln -s %{_libdir}/ruby/gems/%{rb_ver}/gems/%{mod_full_name}/LICENSE %{buildroot}/%{_docdir}/%{name}/LICENSE %post -/usr/sbin/update-alternatives --install \ +%{_sbindir}/update-alternatives --install \ %{_bindir}/hiera hiera %{_bindir}/hiera%{mod_branch} %{mod_weight} %preun if [ "$1" = 0 ] ; then - /usr/sbin/update-alternatives --remove hiera %{_bindir}/hiera%{mod_branch} + %{_sbindir}/update-alternatives --remove hiera %{_bindir}/hiera%{mod_branch} fi %files %defattr(-,root,root,-) -%config(noreplace) %_sysconfdir/hiera.yaml +%config(noreplace) %{_sysconfdir}/hiera.yaml %{_docdir}/%{name} %{_bindir}/hiera%{mod_branch} %ghost %{_bindir}/hiera ++++++ hiera-1.2.1.gem -> hiera-1.3.4.gem ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/README.md new/README.md --- old/README.md 1970-01-01 01:00:00.000000000 +0100 +++ new/README.md 1970-01-01 01:00:00.000000000 +0100 @@ -117,7 +117,7 @@ :logger: console :hierarchy: - - "%{location}" + - "sites/%{location}" - common :yaml: @@ -130,14 +130,14 @@ This configuration will require YAML files in _/etc/puppet/hieradata_ these need to contain Hash data, sample files matching the hierarchy described in the _Why?_ section are below: -_/etc/puppet/hieradata/dc1.yaml_: +_/etc/puppet/hieradata/sites/dc1.yaml_: <pre> --- ntpserver: ntp1.dc1.example.com sysadmin: [email protected] </pre> -_/etc/puppet/hieradata/dc2.yaml_: +_/etc/puppet/hieradata/sites/dc2.yaml_: <pre> --- ntpserver: ntp1.dc2.example.com diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/bin/hiera new/bin/hiera --- old/bin/hiera 1970-01-01 01:00:00.000000000 +0100 +++ new/bin/hiera 1970-01-01 01:00:00.000000000 +0100 @@ -1,5 +1,9 @@ #!/usr/bin/env ruby +# For security reasons, ensure that '.' is not on the load path +# This is primarily for 1.8.7 since 1.9.2+ doesn't put '.' on the load path +$LOAD_PATH.delete '.' + # CLI client for Hiera. # # To lookup the 'release' key for a node given Puppet YAML facts: @@ -24,6 +28,7 @@ require 'hiera' require 'hiera/util' require 'optparse' +require 'pp' options = { :default => nil, @@ -222,5 +227,5 @@ if ans.is_a?(String) puts ans else - p ans + pp ans end Files old/checksums.yaml.gz and new/checksums.yaml.gz differ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/hiera/backend/json_backend.rb new/lib/hiera/backend/json_backend.rb --- old/lib/hiera/backend/json_backend.rb 1970-01-01 01:00:00.000000000 +0100 +++ new/lib/hiera/backend/json_backend.rb 1970-01-01 01:00:00.000000000 +0100 @@ -21,7 +21,7 @@ next unless File.exist?(jsonfile) - data = @cache.read(jsonfile, Hash, {}) do |data| + data = @cache.read_file(jsonfile, Hash) do |data| JSON.parse(data) end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/hiera/backend/yaml_backend.rb new/lib/hiera/backend/yaml_backend.rb --- old/lib/hiera/backend/yaml_backend.rb 1970-01-01 01:00:00.000000000 +0100 +++ new/lib/hiera/backend/yaml_backend.rb 1970-01-01 01:00:00.000000000 +0100 @@ -13,14 +13,9 @@ Hiera.debug("Looking up #{key} in YAML backend") - Backend.datasources(scope, order_override) do |source| - Hiera.debug("Looking for data source #{source}") - yamlfile = Backend.datafile(:yaml, scope, source, "yaml") || next - - next unless File.exist?(yamlfile) - - data = @cache.read(yamlfile, Hash, {}) do |data| - YAML.load(data) + Backend.datasourcefiles(:yaml, scope, "yaml", order_override) do |source, yamlfile| + data = @cache.read_file(yamlfile, Hash) do |data| + YAML.load(data) || {} end next if data.empty? @@ -55,6 +50,12 @@ return answer end + + private + + def file_exists?(path) + File.exist? path + end end end end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/hiera/backend.rb new/lib/hiera/backend.rb --- old/lib/hiera/backend.rb 1970-01-01 01:00:00.000000000 +0100 +++ new/lib/hiera/backend.rb 1970-01-01 01:00:00.000000000 +0100 @@ -1,5 +1,5 @@ require 'hiera/util' -require 'hiera/recursive_lookup' +require 'hiera/interpolate' begin require 'deep_merge' @@ -8,21 +8,25 @@ class Hiera module Backend - INTERPOLATION = /%\{([^\}]*)\}/ - class << self # Data lives in /var/lib/hiera by default. If a backend # supplies a datadir in the config it will be used and # subject to variable expansion based on scope def datadir(backend, scope) backend = backend.to_sym - default = Hiera::Util.var_dir - if Config.include?(backend) - parse_string(Config[backend][:datadir] || default, scope) + if Config[backend] && Config[backend][:datadir] + dir = Config[backend][:datadir] else - parse_string(default, scope) + dir = Hiera::Util.var_dir + end + + if !dir.is_a?(String) + raise(Hiera::InvalidConfigurationError, + "datadir for #{backend} cannot be an array") end + + parse_string(dir, scope) end # Finds the path to a datafile based on the Backend#datadir @@ -30,15 +34,19 @@ # # If the file is not found nil is returned def datafile(backend, scope, source, extension) - file = File.join([datadir(backend, scope), "#{source}.#{extension}"]) + datafile_in(datadir(backend, scope), source, extension) + end - unless File.exist?(file) - Hiera.debug("Cannot find datafile #{file}, skipping") + # @api private + def datafile_in(datadir, source, extension) + file = File.join(datadir, "#{source}.#{extension}") - return nil + if File.exist?(file) + file + else + Hiera.debug("Cannot find datafile #{file}, skipping") + nil end - - return file end # Constructs a list of data sources to search @@ -69,6 +77,35 @@ end end + # Constructs a list of data files to search + # + # If you give it a specific hierarchy it will just use that + # else it will use the global configured one, failing that + # it will just look in the 'common' data source. + # + # An override can be supplied that will be pre-pended to the + # hierarchy. + # + # The source names will be subject to variable expansion based + # on scope + # + # Only files that exist will be returned. If the file is missing, then + # the block will not receive the file. + # + # @yield [String, String] the source string and the name of the resulting file + # @api public + def datasourcefiles(backend, scope, extension, override=nil, hierarchy=nil) + datadir = Backend.datadir(backend, scope) + Backend.datasources(scope, override, hierarchy) do |source| + Hiera.debug("Looking for data source #{source}") + file = datafile_in(datadir, source, extension) + + if file + yield source, file + end + end + end + # Parse a string like <code>'%{foo}'</code> against a supplied # scope and additional scope. If either scope or # extra_scope includes the variable 'foo', then it will @@ -85,23 +122,9 @@ # # @api public def parse_string(data, scope, extra_data={}) - interpolate(data, Hiera::RecursiveLookup.new(scope, extra_data)) + Hiera::Interpolate.interpolate(data, scope, extra_data) end - def interpolate(data, values) - if data.is_a?(String) - data.gsub(INTERPOLATION) do - name = $1 - values.lookup(name) do |value| - interpolate(value, values) - end - end - else - data - end - end - private :interpolate - # Parses a answer received from data files # # Ultimately it just pass the data through parse_string but @@ -114,7 +137,8 @@ elsif data.is_a?(Hash) answer = {} data.each_pair do |key, val| - answer[key] = parse_answer(val, scope, extra_data) + interpolated_key = parse_string(key, scope, extra_data) + answer[interpolated_key] = parse_answer(val, scope, extra_data) end return answer @@ -145,7 +169,7 @@ # Deep merge options use the Hash utility function provided by [deep_merge](https://github.com/peritor/deep_merge) # # :native => Native Hash.merge - # :deep => Use Hash.deep_merge + # :deep => Use Hash.deep_merge # :deeper => Use Hash.deep_merge! # def merge_answer(left,right) @@ -205,6 +229,10 @@ return default if answer.nil? return answer end + + def clear! + @backends = {} + end end end end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/hiera/error.rb new/lib/hiera/error.rb --- old/lib/hiera/error.rb 1970-01-01 01:00:00.000000000 +0100 +++ new/lib/hiera/error.rb 1970-01-01 01:00:00.000000000 +0100 @@ -0,0 +1,4 @@ +class Hiera + class Error < StandardError; end + class InvalidConfigurationError < Error; end +end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/hiera/filecache.rb new/lib/hiera/filecache.rb --- old/lib/hiera/filecache.rb 1970-01-01 01:00:00.000000000 +0100 +++ new/lib/hiera/filecache.rb 1970-01-01 01:00:00.000000000 +0100 @@ -24,32 +24,44 @@ # reading/parsing fails it will return {} instead # # Prior to calling this method you should be sure the file exist - def read(path, expected_type=nil, default=nil) - @cache[path] ||= {:data => nil, :meta => path_metadata(path)} - - if File.exist?(path) && !@cache[path][:data] || stale?(path) - if block_given? - begin - @cache[path][:data] = yield(File.read(path)) - rescue => e - Hiera.debug("Reading data from %s failed: %s: %S" % [path, e.class, e.to_s]) - @cache[path][:data] = default - end - else - @cache[path][:data] = File.read(path) - end + def read(path, expected_type = Object, default=nil, &block) + read_file(path, expected_type, &block) + rescue TypeError => detail + Hiera.debug("#{detail.message}, setting defaults") + @cache[path][:data] = default + rescue => detail + error = "Reading data from #{path} failed: #{detail.class}: #{detail}" + if default.nil? + raise detail + else + Hiera.debug(error) + @cache[path][:data] = default end + end - if block_given? && !expected_type.nil? - unless @cache[path][:data].is_a?(expected_type) - Hiera.debug("Data retrieved from %s is not a %s, setting defaults" % [path, expected_type]) - @cache[path][:data] = default + # Read a file when it changes. If a file is re-read and has not changed since the last time + # then the last, processed, contents will be returned. + # + # The processed data can also be checked against an expected type. If the + # type does not match a TypeError is raised. + # + # No error handling is done inside this method. Any failed reads or errors + # in processing will be propagated to the caller + def read_file(path, expected_type = Object) + if stale?(path) + data = File.read(path) + @cache[path][:data] = block_given? ? yield(data) : data + + if !@cache[path][:data].is_a?(expected_type) + raise TypeError, "Data retrieved from #{path} is #{data.class} not #{expected_type}" end end @cache[path][:data] end + private + def stale?(path) meta = path_metadata(path) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/hiera/interpolate.rb new/lib/hiera/interpolate.rb --- old/lib/hiera/interpolate.rb 1970-01-01 01:00:00.000000000 +0100 +++ new/lib/hiera/interpolate.rb 1970-01-01 01:00:00.000000000 +0100 @@ -0,0 +1,62 @@ +require 'hiera/backend' +require 'hiera/recursive_guard' + +class Hiera::Interpolate + class << self + INTERPOLATION = /%\{([^\}]*)\}/ + METHOD_INTERPOLATION = /%\{(scope|hiera)\(['"]([^"']*)["']\)\}/ + + def interpolate(data, scope, extra_data) + if data.is_a?(String) + # Wrapping do_interpolation in a gsub block ensures we process + # each interpolation site in isolation using separate recursion guards. + data.gsub(INTERPOLATION) do |match| + do_interpolation(match, Hiera::RecursiveGuard.new, scope, extra_data) + end + else + data + end + end + + def do_interpolation(data, recurse_guard, scope, extra_data) + if data.is_a?(String) && (match = data.match(INTERPOLATION)) + interpolation_variable = match[1] + recurse_guard.check(interpolation_variable) do + interpolate_method, key = get_interpolation_method_and_key(data) + interpolated_data = send(interpolate_method, data, key, scope, extra_data) + do_interpolation(interpolated_data, recurse_guard, scope, extra_data) + end + else + data + end + end + private :do_interpolation + + def get_interpolation_method_and_key(data) + if (match = data.match(METHOD_INTERPOLATION)) + case match[1] + when 'hiera' then [:hiera_interpolate, match[2]] + when 'scope' then [:scope_interpolate, match[2]] + end + elsif (match = data.match(INTERPOLATION)) + [:scope_interpolate, match[1]] + end + end + private :get_interpolation_method_and_key + + def scope_interpolate(data, key, scope, extra_data) + value = scope[key] + if value.nil? || value == :undefined + value = extra_data[key] + end + + value + end + private :scope_interpolate + + def hiera_interpolate(data, key, scope, extra_data) + Hiera::Backend.lookup(key, nil, scope, nil, :priority) + end + private :hiera_interpolate + end +end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/hiera/recursive_guard.rb new/lib/hiera/recursive_guard.rb --- old/lib/hiera/recursive_guard.rb 1970-01-01 01:00:00.000000000 +0100 +++ new/lib/hiera/recursive_guard.rb 1970-01-01 01:00:00.000000000 +0100 @@ -0,0 +1,20 @@ +# Allow for safe recursive lookup of values during variable interpolation. +# +# @api private +class Hiera::InterpolationLoop < StandardError; end + +class Hiera::RecursiveGuard + def initialize + @seen = [] + end + + def check(value, &block) + if @seen.include?(value) + raise Hiera::InterpolationLoop, "Detected in [#{@seen.join(', ')}]" + end + @seen.push(value) + ret = yield + @seen.pop + ret + end +end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/hiera/recursive_lookup.rb new/lib/hiera/recursive_lookup.rb --- old/lib/hiera/recursive_lookup.rb 1970-01-01 01:00:00.000000000 +0100 +++ new/lib/hiera/recursive_lookup.rb 1970-01-01 01:00:00.000000000 +0100 @@ -1,31 +0,0 @@ -# Allow for safe recursive lookup of values during variable interpolation. -# -# @api private -class Hiera::RecursiveLookup - def initialize(scope, extra_data) - @seen = [] - @scope = scope - @extra_data = extra_data - end - - def lookup(name, &block) - if @seen.include?(name) - raise Exception, "Interpolation loop detected in [#{@seen.join(', ')}]" - end - @seen.push(name) - ret = yield(current_value) - @seen.pop - ret - end - - def current_value - name = @seen.last - - scope_val = @scope[name] - if scope_val.nil? || scope_val == :undefined - @extra_data[name] - else - scope_val - end - end -end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/hiera/version.rb new/lib/hiera/version.rb --- old/lib/hiera/version.rb 1970-01-01 01:00:00.000000000 +0100 +++ new/lib/hiera/version.rb 1970-01-01 01:00:00.000000000 +0100 @@ -0,0 +1,89 @@ +# The version method and constant are isolated in hiera/version.rb so that a +# simple `require 'hiera/version'` allows a rubygems gemspec or bundler +# Gemfile to get the hiera version of the gem install. +# +# The version is programatically settable because we want to allow the +# Raketasks and such to set the version based on the output of `git describe` + + +class Hiera + VERSION = "1.3.4" + + ## + # version is a public API method intended to always provide a fast and + # lightweight way to determine the version of hiera. + # + # The intent is that software external to hiera be able to determine the + # hiera version with no side-effects. The expected use is: + # + # require 'hiera/version' + # version = Hiera.version + # + # This function has the following ordering precedence. This precedence list + # is designed to facilitate automated packaging tasks by simply writing to + # the VERSION file in the same directory as this source file. + # + # 1. If a version has been explicitly assigned using the Hiera.version= + # method, return that version. + # 2. If there is a VERSION file, read the contents, trim any + # trailing whitespace, and return that version string. + # 3. Return the value of the Hiera::VERSION constant hard-coded into + # the source code. + # + # If there is no VERSION file, the method must return the version string of + # the nearest parent version that is an officially released version. That is + # to say, if a branch named 3.1.x contains 25 patches on top of the most + # recent official release of 3.1.1, then the version method must return the + # string "3.1.1" if no "VERSION" file is present. + # + # By design the version identifier is _not_ intended to vary during the life + # a process. There is no guarantee provided that writing to the VERSION file + # while a Hiera process is running will cause the version string to be + # updated. On the contrary, the contents of the VERSION are cached to reduce + # filesystem accesses. + # + # The VERSION file is intended to be used by package maintainers who may be + # applying patches or otherwise changing the software version in a manner + # that warrants a different software version identifier. The VERSION file is + # intended to be managed and owned by the release process and packaging + # related tasks, and as such should not reside in version control. The + # VERSION constant is intended to be version controlled in history. + # + # Ideally, this behavior will allow package maintainers to precisely specify + # the version of the software they're packaging as in the following example: + # + # $ git describe --match "1.2.*" > lib/hiera/VERSION + # $ ruby -r hiera/version -e 'puts Hiera.version' + # 1.2.1-9-g9fda440 + # + # @api public + # + # @return [String] containing the hiera version, e.g. "1.2.1" + def self.version + version_file = File.join(File.dirname(__FILE__), 'VERSION') + return @hiera_version if @hiera_version + if version = read_version_file(version_file) + @hiera_version = version + end + @hiera_version ||= VERSION + end + + def self.version=(version) + @hiera_version = version + end + + ## + # read_version_file reads the content of the "VERSION" file that lives in the + # same directory as this source code file. + # + # @api private + # + # @return [String] for example: "1.6.14-6-gea42046" or nil if the VERSION + # file does not exist. + def self.read_version_file(path) + if File.exists?(path) + File.read(path).chomp + end + end + private_class_method :read_version_file +end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/hiera.rb new/lib/hiera.rb --- old/lib/hiera.rb 1970-01-01 01:00:00.000000000 +0100 +++ new/lib/hiera.rb 1970-01-01 01:00:00.000000000 +0100 @@ -1,8 +1,8 @@ require 'yaml' class Hiera - VERSION = "1.2.1" - + require "hiera/error" + require "hiera/version" require "hiera/config" require "hiera/util" require "hiera/backend" @@ -15,10 +15,6 @@ class << self attr_reader :logger - def version - VERSION - end - # Loggers are pluggable, just provide a class called # Hiera::Foo_logger and respond to :warn and :debug # diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/metadata new/metadata --- old/metadata 1970-01-01 01:00:00.000000000 +0100 +++ new/metadata 1970-01-01 01:00:00.000000000 +0100 @@ -1,114 +1,100 @@ ---- !ruby/object:Gem::Specification +--- !ruby/object:Gem::Specification name: hiera -version: !ruby/object:Gem::Version - hash: 29 - prerelease: - segments: - - 1 - - 2 - - 1 - version: 1.2.1 +version: !ruby/object:Gem::Version + version: 1.3.4 platform: ruby -authors: +authors: - Puppet Labs autorequire: bindir: bin cert_chain: [] - -date: 2013-04-18 00:00:00 Z -dependencies: -- !ruby/object:Gem::Dependency +date: 2014-06-06 00:00:00.000000000 Z +dependencies: +- !ruby/object:Gem::Dependency name: json_pure - prerelease: false - requirement: &id001 !ruby/object:Gem::Requirement - none: false - requirements: - - - ">=" - - !ruby/object:Gem::Version - hash: 3 - segments: - - 0 - version: "0" + requirement: !ruby/object:Gem::Requirement + requirements: + - - '>=' + - !ruby/object:Gem::Version + version: '0' type: :runtime - version_requirements: *id001 + prerelease: false + version_requirements: !ruby/object:Gem::Requirement + requirements: + - - '>=' + - !ruby/object:Gem::Version + version: '0' description: A pluggable data store for hierarcical data email: [email protected] -executables: +executables: - hiera extensions: [] - extra_rdoc_files: [] - -files: +files: - bin/hiera -- lib/hiera/puppet_logger.rb -- lib/hiera/recursive_lookup.rb +- lib/hiera/backend.rb +- lib/hiera/backend/json_backend.rb +- lib/hiera/backend/yaml_backend.rb +- lib/hiera/config.rb - lib/hiera/console_logger.rb -- lib/hiera/filecache.rb +- lib/hiera/error.rb - lib/hiera/fallback_logger.rb -- lib/hiera/util.rb -- lib/hiera/backend.rb +- lib/hiera/filecache.rb +- lib/hiera/interpolate.rb - lib/hiera/noop_logger.rb -- lib/hiera/config.rb -- lib/hiera/backend/yaml_backend.rb -- lib/hiera/backend/json_backend.rb +- lib/hiera/puppet_logger.rb +- lib/hiera/recursive_guard.rb +- lib/hiera/util.rb +- lib/hiera/version.rb - lib/hiera.rb - COPYING - README.md - LICENSE -- spec/unit/util_spec.rb -- spec/unit/puppet_logger_spec.rb -- spec/unit/config_spec.rb +- spec/spec_helper.rb +- spec/unit/backend/json_backend_spec.rb +- spec/unit/backend/yaml_backend_spec.rb - spec/unit/backend_spec.rb -- spec/unit/filecache_spec.rb -- spec/unit/hiera_spec.rb +- spec/unit/config_spec.rb - spec/unit/console_logger_spec.rb -- spec/unit/backend/yaml_backend_spec.rb -- spec/unit/backend/json_backend_spec.rb - spec/unit/fallback_logger_spec.rb -- spec/spec_helper.rb +- spec/unit/filecache_spec.rb +- spec/unit/hiera_spec.rb +- spec/unit/puppet_logger_spec.rb +- spec/unit/util_spec.rb +- spec/unit/version_spec.rb homepage: https://github.com/puppetlabs/hiera licenses: [] - +metadata: {} post_install_message: rdoc_options: [] - -require_paths: +require_paths: - lib -required_ruby_version: !ruby/object:Gem::Requirement - none: false - requirements: - - - ">=" - - !ruby/object:Gem::Version - hash: 3 - segments: - - 0 - version: "0" -required_rubygems_version: !ruby/object:Gem::Requirement - none: false - requirements: - - - ">=" - - !ruby/object:Gem::Version - hash: 3 - segments: - - 0 - version: "0" +required_ruby_version: !ruby/object:Gem::Requirement + requirements: + - - '>=' + - !ruby/object:Gem::Version + version: '0' +required_rubygems_version: !ruby/object:Gem::Requirement + requirements: + - - '>=' + - !ruby/object:Gem::Version + version: '0' requirements: [] - rubyforge_project: -rubygems_version: 1.8.24 +rubygems_version: 2.0.3 signing_key: -specification_version: 3 +specification_version: 4 summary: Light weight hierarchical data store -test_files: -- spec/unit/util_spec.rb -- spec/unit/puppet_logger_spec.rb -- spec/unit/config_spec.rb +test_files: +- spec/spec_helper.rb +- spec/unit/backend/json_backend_spec.rb +- spec/unit/backend/yaml_backend_spec.rb - spec/unit/backend_spec.rb -- spec/unit/filecache_spec.rb -- spec/unit/hiera_spec.rb +- spec/unit/config_spec.rb - spec/unit/console_logger_spec.rb -- spec/unit/backend/yaml_backend_spec.rb -- spec/unit/backend/json_backend_spec.rb - spec/unit/fallback_logger_spec.rb -- spec/spec_helper.rb +- spec/unit/filecache_spec.rb +- spec/unit/hiera_spec.rb +- spec/unit/puppet_logger_spec.rb +- spec/unit/util_spec.rb +- spec/unit/version_spec.rb diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/spec/spec_helper.rb new/spec/spec_helper.rb --- old/spec/spec_helper.rb 1970-01-01 01:00:00.000000000 +0100 +++ new/spec/spec_helper.rb 1970-01-01 01:00:00.000000000 +0100 @@ -8,6 +8,17 @@ RSpec.configure do |config| config.mock_with :mocha + + config.after :suite do + # Log the spec order to a file, but only if the LOG_SPEC_ORDER environment variable is + # set. This should be enabled on Jenkins runs, as it can be used with Nick L.'s bisect + # script to help identify and debug order-dependent spec failures. + if ENV['LOG_SPEC_ORDER'] + File.open("./spec_order.txt", "w") do |logfile| + config.instance_variable_get(:@files_to_run).each { |f| logfile.puts f } + end + end + end end # In ruby 1.8.5 Dir does not have mktmpdir defined, so this monkey patches diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/spec/unit/backend/json_backend_spec.rb new/spec/unit/backend/json_backend_spec.rb --- old/spec/unit/backend/json_backend_spec.rb 1970-01-01 01:00:00.000000000 +0100 +++ new/spec/unit/backend/json_backend_spec.rb 1970-01-01 01:00:00.000000000 +0100 @@ -33,7 +33,7 @@ Backend.expects(:datafile).with(:json, {}, "one", "json").returns("/nonexisting/one.json").times(3) File.stubs(:exist?).with("/nonexisting/one.json").returns(true) - @cache.expects(:read).with("/nonexisting/one.json", Hash, {}).returns({"stringval" => "string", "boolval" => true, "numericval" => 1}).times(3) + @cache.expects(:read_file).with("/nonexisting/one.json", Hash).returns({"stringval" => "string", "boolval" => true, "numericval" => 1}).times(3) @backend.lookup("stringval", {}, nil, :priority).should == "string" @backend.lookup("boolval", {}, nil, :priority).should == true @@ -47,7 +47,7 @@ Backend.expects(:datafile).with(:json, scope, "two", "json").never File.stubs(:exist?).with("/nonexisting/one.json").returns(true) - @cache.expects(:read).with("/nonexisting/one.json", Hash, {}).returns({"key" => "test_%{rspec}"}) + @cache.expects(:read_file).with("/nonexisting/one.json", Hash).returns({"key" => "test_%{rspec}"}) @backend.lookup("key", scope, nil, :priority).should == "test_test" end @@ -63,8 +63,8 @@ File.expects(:exist?).with("/nonexisting/one.json").returns(true) File.expects(:exist?).with("/nonexisting/two.json").returns(true) - @cache.expects(:read).with("/nonexisting/one.json", Hash, {}).returns({"key" => "answer"}) - @cache.expects(:read).with("/nonexisting/two.json", Hash, {}).returns({"key" => "answer"}) + @cache.expects(:read_file).with("/nonexisting/one.json", Hash).returns({"key" => "answer"}) + @cache.expects(:read_file).with("/nonexisting/two.json", Hash).returns({"key" => "answer"}) @backend.lookup("key", {}, nil, :array).should == ["answer", "answer"] end @@ -75,7 +75,7 @@ Backend.expects(:datafile).with(:json, {"rspec" => "test"}, "one", "json").returns("/nonexisting/one.json") File.expects(:exist?).with("/nonexisting/one.json").returns(true) - @cache.expects(:read).with("/nonexisting/one.json", Hash, {}).returns({"key" => "test_%{rspec}"}) + @cache.expects(:read_file).with("/nonexisting/one.json", Hash).returns({"key" => "test_%{rspec}"}) @backend.lookup("key", {"rspec" => "test"}, nil, :priority).should == "test_test" end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/spec/unit/backend/yaml_backend_spec.rb new/spec/unit/backend/yaml_backend_spec.rb --- old/spec/unit/backend/yaml_backend_spec.rb 1970-01-01 01:00:00.000000000 +0100 +++ new/spec/unit/backend/yaml_backend_spec.rb 1970-01-01 01:00:00.000000000 +0100 @@ -3,12 +3,29 @@ class Hiera module Backend + class FakeCache + attr_accessor :value + def read(path, expected_type, default, &block) + read_file(path, expected_type, &block) + rescue => e + default + end + + def read_file(path, expected_type, &block) + output = block.call(@value) + if !output.is_a? expected_type + raise TypeError + end + output + end + end + describe Yaml_backend do before do Config.load({}) Hiera.stubs(:debug) Hiera.stubs(:warn) - @cache = mock + @cache = FakeCache.new @backend = Yaml_backend.new(@cache) end @@ -20,124 +37,91 @@ end describe "#lookup" do - it "should look for data in all sources" do - Backend.expects(:datasources).multiple_yields(["one"], ["two"]) - Backend.expects(:datafile).with(:yaml, {}, "one", "yaml").returns(nil) - Backend.expects(:datafile).with(:yaml, {}, "two", "yaml").returns(nil) - - @backend.lookup("key", {}, nil, :priority) - end - it "should pick data earliest source that has it for priority searches" do - Backend.expects(:datasources).multiple_yields(["one"], ["two"]) - Backend.expects(:datafile).with(:yaml, {}, "one", "yaml").returns("/nonexisting/one.yaml") - Backend.expects(:datafile).with(:yaml, {}, "two", "yaml").returns(nil).never - @cache.expects(:read).with("/nonexisting/one.yaml", Hash, {}).returns({"key"=>"answer"}) - File.stubs(:exist?).with("/nonexisting/one.yaml").returns(true) + Backend.expects(:datasourcefiles).with(:yaml, {}, "yaml", nil).yields(["one", "/nonexisting/one.yaml"]) + @cache.value = "---\nkey: answer" @backend.lookup("key", {}, nil, :priority).should == "answer" end - it "should not look up missing data files" do - Backend.expects(:datasources).multiple_yields(["one"]) - Backend.expects(:datafile).with(:yaml, {}, "one", "yaml").returns(nil) - YAML.expects(:load_file).never - - @backend.lookup("key", {}, nil, :priority) - end - - it "should return nil for empty data files" do - Backend.expects(:datasources).multiple_yields(["one"]) - Backend.expects(:datafile).with(:yaml, {}, "one", "yaml").returns("/nonexisting/one.yaml") - File.stubs(:exist?).with("/nonexisting/one.yaml").returns(true) - @cache.expects(:read).with("/nonexisting/one.yaml", Hash, {}).returns({}) - - @backend.lookup("key", {}, nil, :priority).should be_nil + describe "handling unexpected YAML values" do + before do + Backend.expects(:datasourcefiles).with(:yaml, {}, "yaml", nil).yields(["one", "/nonexisting/one.yaml"]) + end + + it "returns nil when the YAML value is nil" do + @cache.value = "---\n" + @backend.lookup("key", {}, nil, :priority).should be_nil + end + + it "returns nil when the YAML file is false" do + @cache.value = "" + @backend.lookup("key", {}, nil, :priority).should be_nil + end + + it "raises a TypeError when the YAML value is not a hash" do + @cache.value = "---\n[one, two, three]" + expect { @backend.lookup("key", {}, nil, :priority) }.to raise_error(TypeError) + end end it "should build an array of all data sources for array searches" do - Backend.expects(:datasources).multiple_yields(["one"], ["two"]) - Backend.expects(:datafile).with(:yaml, {}, "one", "yaml").returns("/nonexisting/one.yaml") - Backend.expects(:datafile).with(:yaml, {}, "two", "yaml").returns("/nonexisting/two.yaml") - File.stubs(:exist?).with("/nonexisting/one.yaml").returns(true) - File.stubs(:exist?).with("/nonexisting/two.yaml").returns(true) - - @cache.expects(:read).with("/nonexisting/one.yaml", Hash, {}).returns({"key"=>"answer"}) - @cache.expects(:read).with("/nonexisting/two.yaml", Hash, {}).returns({"key"=>"answer"}) + Backend.expects(:datasourcefiles).with(:yaml, {}, "yaml", nil).multiple_yields(["one", "/nonexisting/one.yaml"], ["two", "/nonexisting/two.yaml"]) + @cache.expects(:read_file).with("/nonexisting/one.yaml", Hash).returns({"key"=>"answer"}) + @cache.expects(:read_file).with("/nonexisting/two.yaml", Hash).returns({"key"=>"answer"}) @backend.lookup("key", {}, nil, :array).should == ["answer", "answer"] end it "should ignore empty hash of data sources for hash searches" do - Backend.expects(:datasources).multiple_yields(["one"], ["two"]) - Backend.expects(:datafile).with(:yaml, {}, "one", "yaml").returns("/nonexisting/one.yaml") - Backend.expects(:datafile).with(:yaml, {}, "two", "yaml").returns("/nonexisting/two.yaml") - File.stubs(:exist?).with("/nonexisting/one.yaml").returns(true) - File.stubs(:exist?).with("/nonexisting/two.yaml").returns(true) + Backend.expects(:datasourcefiles).with(:yaml, {}, "yaml", nil).multiple_yields(["one", "/nonexisting/one.yaml"], ["two", "/nonexisting/two.yaml"]) - @cache.expects(:read).with("/nonexisting/one.yaml", Hash, {}).returns({}) - @cache.expects(:read).with("/nonexisting/two.yaml", Hash, {}).returns({"key"=>{"a"=>"answer"}}) + @cache.expects(:read_file).with("/nonexisting/one.yaml", Hash).returns({}) + @cache.expects(:read_file).with("/nonexisting/two.yaml", Hash).returns({"key"=>{"a"=>"answer"}}) @backend.lookup("key", {}, nil, :hash).should == {"a" => "answer"} end it "should build a merged hash of data sources for hash searches" do - Backend.expects(:datasources).multiple_yields(["one"], ["two"]) - Backend.expects(:datafile).with(:yaml, {}, "one", "yaml").returns("/nonexisting/one.yaml") - Backend.expects(:datafile).with(:yaml, {}, "two", "yaml").returns("/nonexisting/two.yaml") - File.stubs(:exist?).with("/nonexisting/one.yaml").returns(true) - File.stubs(:exist?).with("/nonexisting/two.yaml").returns(true) + Backend.expects(:datasourcefiles).with(:yaml, {}, "yaml", nil).multiple_yields(["one", "/nonexisting/one.yaml"], ["two", "/nonexisting/two.yaml"]) - @cache.expects(:read).with("/nonexisting/one.yaml", Hash, {}).returns({"key"=>{"a"=>"answer"}}) - @cache.expects(:read).with("/nonexisting/two.yaml", Hash, {}).returns({"key"=>{"b"=>"answer", "a"=>"wrong"}}) + @cache.expects(:read_file).with("/nonexisting/one.yaml", Hash).returns({"key"=>{"a"=>"answer"}}) + @cache.expects(:read_file).with("/nonexisting/two.yaml", Hash).returns({"key"=>{"b"=>"answer", "a"=>"wrong"}}) @backend.lookup("key", {}, nil, :hash).should == {"a" => "answer", "b" => "answer"} end it "should fail when trying to << a Hash" do - Backend.expects(:datasources).multiple_yields(["one"], ["two"]) - Backend.expects(:datafile).with(:yaml, {}, "one", "yaml").returns("/nonexisting/one.yaml") - Backend.expects(:datafile).with(:yaml, {}, "two", "yaml").returns("/nonexisting/two.yaml") - File.stubs(:exist?).with("/nonexisting/one.yaml").returns(true) - File.stubs(:exist?).with("/nonexisting/two.yaml").returns(true) + Backend.expects(:datasourcefiles).with(:yaml, {}, "yaml", nil).multiple_yields(["one", "/nonexisting/one.yaml"], ["two", "/nonexisting/two.yaml"]) - @cache.expects(:read).with("/nonexisting/one.yaml", Hash, {}).returns({"key"=>["a", "answer"]}) - @cache.expects(:read).with("/nonexisting/two.yaml", Hash, {}).returns({"key"=>{"a"=>"answer"}}) + @cache.expects(:read_file).with("/nonexisting/one.yaml", Hash).returns({"key"=>["a", "answer"]}) + @cache.expects(:read_file).with("/nonexisting/two.yaml", Hash).returns({"key"=>{"a"=>"answer"}}) expect {@backend.lookup("key", {}, nil, :array)}.to raise_error(Exception, "Hiera type mismatch: expected Array and got Hash") end it "should fail when trying to merge an Array" do - Backend.expects(:datasources).multiple_yields(["one"], ["two"]) - Backend.expects(:datafile).with(:yaml, {}, "one", "yaml").returns("/nonexisting/one.yaml") - Backend.expects(:datafile).with(:yaml, {}, "two", "yaml").returns("/nonexisting/two.yaml") - File.stubs(:exist?).with("/nonexisting/one.yaml").returns(true) - File.stubs(:exist?).with("/nonexisting/two.yaml").returns(true) + Backend.expects(:datasourcefiles).with(:yaml, {}, "yaml", nil).multiple_yields(["one", "/nonexisting/one.yaml"], ["two", "/nonexisting/two.yaml"]) - @cache.expects(:read).with("/nonexisting/one.yaml", Hash, {}).returns({"key"=>{"a"=>"answer"}}) - @cache.expects(:read).with("/nonexisting/two.yaml", Hash, {}).returns({"key"=>["a", "wrong"]}) + @cache.expects(:read_file).with("/nonexisting/one.yaml", Hash).returns({"key"=>{"a"=>"answer"}}) + @cache.expects(:read_file).with("/nonexisting/two.yaml", Hash).returns({"key"=>["a", "wrong"]}) expect { @backend.lookup("key", {}, nil, :hash) }.to raise_error(Exception, "Hiera type mismatch: expected Hash and got Array") end it "should parse the answer for scope variables" do - Backend.expects(:datasources).yields("one") - Backend.expects(:datafile).with(:yaml, {"rspec" => "test"}, "one", "yaml").returns("/nonexisting/one.yaml") - File.stubs(:exist?).with("/nonexisting/one.yaml").returns(true) + Backend.expects(:datasourcefiles).with(:yaml, {"rspec" => "test"}, "yaml", nil).multiple_yields(["one", "/nonexisting/one.yaml"]) - @cache.expects(:read).with("/nonexisting/one.yaml", Hash, {}).returns({"key"=>"test_%{rspec}"}) + @cache.expects(:read_file).with("/nonexisting/one.yaml", Hash).returns({"key"=>"test_%{rspec}"}) @backend.lookup("key", {"rspec" => "test"}, nil, :priority).should == "test_test" end it "should retain datatypes found in yaml files" do - Backend.expects(:datasources).yields("one").times(3) - Backend.expects(:datafile).with(:yaml, {}, "one", "yaml").returns("/nonexisting/one.yaml").times(3) - File.stubs(:exist?).with("/nonexisting/one.yaml").returns(true) + Backend.expects(:datasourcefiles).with(:yaml, {}, "yaml", nil).multiple_yields(["one", "/nonexisting/one.yaml"]).times(3) - yaml = "---\nstringval: 'string'\nboolval: true\nnumericval: 1" - @cache.expects(:read).with("/nonexisting/one.yaml", Hash, {}).times(3).returns({"boolval"=>true, "numericval"=>1, "stringval"=>"string"}) + @cache.value = "---\nstringval: 'string'\nboolval: true\nnumericval: 1" @backend.lookup("stringval", {}, nil, :priority).should == "string" @backend.lookup("boolval", {}, nil, :priority).should == true diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/spec/unit/backend_spec.rb new/spec/unit/backend_spec.rb --- old/spec/unit/backend_spec.rb 1970-01-01 01:00:00.000000000 +0100 +++ new/spec/unit/backend_spec.rb 1970-01-01 01:00:00.000000000 +0100 @@ -5,15 +5,30 @@ describe Backend do describe "#datadir" do it "interpolates any values in the configured value" do - Config.load({:rspec => {:datadir => "/tmp"}}) - Backend.expects(:parse_string).with("/tmp", {}) - Backend.datadir(:rspec, {}) + Config.load({:rspec => {:datadir => "/tmp/%{interpolate}"}}) + + dir = Backend.datadir(:rspec, { "interpolate" => "my_data" }) + + dir.should == "/tmp/my_data" end it "defaults to a directory in var" do Config.load({}) - Backend.expects(:parse_string).with(Hiera::Util.var_dir, {}) - Backend.datadir(:rspec, {}) + Backend.datadir(:rspec, {}).should == Hiera::Util.var_dir + + Config.load({:rspec => nil}) + Backend.datadir(:rspec, {}).should == Hiera::Util.var_dir + + Config.load({:rspec => {}}) + Backend.datadir(:rspec, {}).should == Hiera::Util.var_dir + end + + it "fails when the datadir is an array" do + Config.load({:rspec => {:datadir => []}}) + + expect do + Backend.datadir(:rspec, {}) + end.to raise_error(Hiera::InvalidConfigurationError, /datadir for rspec cannot be an array/) end end @@ -103,11 +118,19 @@ Backend.parse_string(input, {}).should == input end - it "replaces interpolations with data looked up in the scope" do - input = "replace %{part1} and %{part2}" - scope = {"part1" => "value of part1", "part2" => "value of part2"} + @scope_interpolation_tests = { + "replace %{part1} and %{part2}" => + "replace value of part1 and value of part2", + "replace %{scope('part1')} and %{scope('part2')}" => + "replace value of part1 and value of part2" + } + + @scope_interpolation_tests.each do |input, expected| + it "replaces interpolations with data looked up in the scope" do + scope = {"part1" => "value of part1", "part2" => "value of part2"} - Backend.parse_string(input, scope).should == "replace value of part1 and value of part2" + Backend.parse_string(input, scope).should == expected + end end it "replaces interpolations with data looked up in extra_data when scope does not contain the value" do @@ -120,14 +143,27 @@ Backend.parse_string(input, {"rspec" => "test"}, {"rspec" => "fail"}).should == "test_test_test" end - it "interprets nil in scope as a non-value" do - input = "test_%{rspec}_test" - Backend.parse_string(input, {"rspec" => nil}).should == "test__test" + @interprets_nil_in_scope_tests = { + "test_%{rspec}_test" => "test__test", + "test_%{scope('rspec')}_test" => "test__test" + } + + @interprets_nil_in_scope_tests.each do |input, expected| + it "interprets nil in scope as a non-value" do + Backend.parse_string(input, {"rspec" => nil}).should == expected + end end - it "interprets false in scope as a real value" do - input = "test_%{rspec}_test" - Backend.parse_string(input, {"rspec" => false}).should == "test_false_test" + @interprets_false_in_scope_tests = { + "test_%{rspec}_test" => "test_false_test", + "test_%{scope('rspec')}_test" => "test_false_test" + } + + @interprets_false_in_scope_tests.each do |input, expected| + it "interprets false in scope as a real value" do + input = "test_%{scope('rspec')}_test" + Backend.parse_string(input, {"rspec" => false}).should == expected + end end it "interprets false in extra_data as a real value" do @@ -140,9 +176,15 @@ Backend.parse_string(input, {}, {"rspec" => nil}).should == "test__test" end - it "interprets :undefined in scope as a non-value" do - input = "test_%{rspec}_test" - Backend.parse_string(input, {"rspec" => :undefined}).should == "test__test" + @interprets_undefined_in_scope_tests = { + "test_%{rspec}_test" => "test__test", + "test_%{scope('rspec')}_test" => "test__test" + } + + @interprets_undefined_in_scope_tests.each do |input, expected| + it "interprets :undefined in scope as a non-value" do + Backend.parse_string(input, {"rspec" => :undefined}).should == expected + end end it "uses the value from extra_data when scope is :undefined" do @@ -150,24 +192,51 @@ Backend.parse_string(input, {"rspec" => :undefined}, { "rspec" => "extra" }).should == "test_extra_test" end - it "looks up the interpolated value exactly as it appears in the input" do - input = "test_%{::rspec::data}_test" - Backend.parse_string(input, {"::rspec::data" => "value"}).should == "test_value_test" + @exact_lookup_tests = { + "test_%{::rspec::data}_test" => "test_value_test", + "test_%{scope('::rspec::data')}_test" => "test_value_test" + } + + @exact_lookup_tests.each do |input, expected| + it "looks up the interpolated value exactly as it appears in the input" do + Backend.parse_string(input, {"::rspec::data" => "value"}).should == expected + end end - it "does not remove any surrounding whitespace when parsing the key to lookup" do - input = "test_%{\trspec::data }_test" - Backend.parse_string(input, {"\trspec::data " => "value"}).should == "test_value_test" + @surrounding_whitespace_tests = { + "test_%{\trspec::data }_test" => "test_value_test", + "test_%{scope('\trspec::data ')}_test" => "test_value_test" + } + @surrounding_whitespace_tests.each do |input, expected| + it "does not remove any surrounding whitespace when parsing the key to lookup" do + Backend.parse_string(input, {"\trspec::data " => "value"}).should == expected + end end - it "does not try removing leading :: when a full lookup fails (#17434)" do - input = "test_%{::rspec::data}_test" - Backend.parse_string(input, {"rspec::data" => "value"}).should == "test__test" + @leading_double_colon_tests = { + "test_%{::rspec::data}_test" => "test__test", + "test_%{scope('::rspec::data')}_test" => "test__test" + } + + @leading_double_colon_tests.each do |input, expected| + it "does not try removing leading :: when a full lookup fails (#17434)" do + Backend.parse_string(input, {"rspec::data" => "value"}).should == expected + end end - it "does not try removing leading sections separated by :: when a full lookup fails (#17434)" do - input = "test_%{::rspec::data}_test" - Backend.parse_string(input, {"data" => "value"}).should == "test__test" + @double_colon_key_tests = { + "test_%{::rspec::data}_test" => "test__test", + "test_%{scope('::rspec::data')}_test" => "test__test" + } + @double_colon_key_tests.each do |input, expected| + it "does not try removing leading sections separated by :: when a full lookup fails (#17434)" do + Backend.parse_string(input, {"data" => "value"}).should == expected + end + end + + it "does not try removing unknown, preceeding characters when looking up values" do + input = "test_%{$var}_test" + Backend.parse_string(input, {"$var" => "value"}).should == "test_value_test" end it "looks up recursively" do @@ -181,7 +250,23 @@ input = "test_%{first}_test" expect do Backend.parse_string(input, scope) - end.to raise_error Exception, "Interpolation loop detected in [first, second]" + end.to raise_error Hiera::InterpolationLoop, "Detected in [first, second]" + end + + it "replaces repeated occurances of the same lookup" do + scope = {"rspec" => "value"} + input = "it replaces %{rspec} and %{rspec}" + Backend.parse_string(input, scope).should == "it replaces value and value" + end + + it "replaces hiera interpolations with data looked up in hiera" do + input = "%{hiera('key1')}" + scope = {} + Config.load({:yaml => {:datadir => "/tmp"}}) + Config.load_backends + Backend::Yaml_backend.any_instance.stubs(:lookup).with("key1", scope, nil, :priority).returns("answer") + + Backend.parse_string(input, scope).should == "answer" end end @@ -201,11 +286,93 @@ Backend.parse_answer(input, {"rspec" => "test"}).should == {"foo"=>"test_test_test", "bar"=>"test_test_test"} end + it "interpolates string in hash keys" do + input = {"%{rspec}" => "test"} + Backend.parse_answer(input, {"rspec" => "foo"}).should == {"foo"=>"test"} + end + + it "interpolates strings in nested hash keys" do + input = {"topkey" => {"%{rspec}" => "test"}} + Backend.parse_answer(input, {"rspec" => "foo"}).should == {"topkey"=>{"foo" => "test"}} + end + it "interpolates strings in a mixed structure of arrays and hashes" do input = {"foo" => "test_%{rspec}_test", "bar" => ["test_%{rspec}_test", "test_%{rspec}_test"]} Backend.parse_answer(input, {"rspec" => "test"}).should == {"foo"=>"test_test_test", "bar"=>["test_test_test", "test_test_test"]} end + it "interpolates hiera lookups values in strings" do + input = "test_%{hiera('rspec')}_test" + scope = {} + Config.load({:yaml => {:datadir => "/tmp"}}) + Config.load_backends + Backend::Yaml_backend.any_instance.stubs(:lookup).with("rspec", scope, nil, :priority).returns("test") + Backend.parse_answer(input, scope).should == "test_test_test" + end + + it "interpolates hiera lookups in each string in an array" do + input = ["test_%{hiera('rspec')}_test", "test_%{hiera('rspec')}_test", ["test_%{hiera('rspec')}_test"]] + scope = {} + Config.load({:yaml => {:datadir => "/tmp"}}) + Config.load_backends + Backend::Yaml_backend.any_instance.stubs(:lookup).with("rspec", scope, nil, :priority).returns("test") + Backend.parse_answer(input, scope).should == ["test_test_test", "test_test_test", ["test_test_test"]] + end + + it "interpolates hiera lookups in each string in a hash" do + input = {"foo" => "test_%{hiera('rspec')}_test", "bar" => "test_%{hiera('rspec')}_test"} + scope = {} + Config.load({:yaml => {:datadir => "/tmp"}}) + Config.load_backends + Backend::Yaml_backend.any_instance.stubs(:lookup).with("rspec", scope, nil, :priority).returns("test") + Backend.parse_answer(input, scope).should == {"foo"=>"test_test_test", "bar"=>"test_test_test"} + end + + it "interpolates hiera lookups in string in hash keys" do + input = {"%{hiera('rspec')}" => "test"} + scope = {} + Config.load({:yaml => {:datadir => "/tmp"}}) + Config.load_backends + Backend::Yaml_backend.any_instance.stubs(:lookup).with("rspec", scope, nil, :priority).returns("foo") + Backend.parse_answer(input, scope).should == {"foo"=>"test"} + end + + it "interpolates hiera lookups in strings in nested hash keys" do + input = {"topkey" => {"%{hiera('rspec')}" => "test"}} + scope = {} + Config.load({:yaml => {:datadir => "/tmp"}}) + Config.load_backends + Backend::Yaml_backend.any_instance.stubs(:lookup).with("rspec", scope, nil, :priority).returns("foo") + Backend.parse_answer(input, scope).should == {"topkey"=>{"foo" => "test"}} + end + + it "interpolates hiera lookups in strings in a mixed structure of arrays and hashes" do + input = {"foo" => "test_%{hiera('rspec')}_test", "bar" => ["test_%{hiera('rspec')}_test", "test_%{hiera('rspec')}_test"]} + scope = {} + Config.load({:yaml => {:datadir => "/tmp"}}) + Config.load_backends + Backend::Yaml_backend.any_instance.stubs(:lookup).with("rspec", scope, nil, :priority).returns("test") + Backend.parse_answer(input, scope).should == {"foo"=>"test_test_test", "bar"=>["test_test_test", "test_test_test"]} + end + + it "interpolates hiera lookups and scope lookups in the same string" do + input = {"foo" => "test_%{hiera('rspec')}_test", "bar" => "test_%{rspec2}_test"} + scope = {"rspec2" => "scope_rspec"} + Config.load({:yaml => {:datadir => "/tmp"}}) + Config.load_backends + Backend::Yaml_backend.any_instance.stubs(:lookup).with("rspec", scope, nil, :priority).returns("hiera_rspec") + Backend.parse_answer(input, scope).should == {"foo"=>"test_hiera_rspec_test", "bar"=>"test_scope_rspec_test"} + end + + it "interpolates hiera and scope lookups with the same lookup query in a single string" do + input = "test_%{hiera('rspec')}_test_%{rspec}" + scope = {"rspec" => "scope_rspec"} + Config.load({:yaml => {:datadir => "/tmp"}}) + Config.load_backends + Backend::Yaml_backend.any_instance.stubs(:lookup).with("rspec", scope, nil, :priority).returns("hiera_rspec") + Backend.parse_answer(input, scope).should == "test_hiera_rspec_test_scope_rspec" + end + it "passes integers unchanged" do input = 1 Backend.parse_answer(input, {"rspec" => "test"}).should == 1 @@ -225,6 +392,12 @@ input = false Backend.parse_answer(input, {"rspec" => "test"}).should == false end + + it "interpolates lookups using single or double quotes" do + input = "test_%{scope(\"rspec\")}_test_%{scope('rspec')}" + scope = {"rspec" => "scope_rspec"} + Backend.parse_answer(input, scope).should == "test_scope_rspec_test_scope_rspec" + end end describe "#resolve_answer" do @@ -244,6 +417,7 @@ end it "caches loaded backends" do + Backend.clear! Hiera.expects(:debug).with(regexp_matches(/Hiera YAML backend starting/)).once Config.load({:yaml => {:datadir => "/tmp"}}) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/spec/unit/config_spec.rb new/spec/unit/config_spec.rb --- old/spec/unit/config_spec.rb 1970-01-01 01:00:00.000000000 +0100 +++ new/spec/unit/config_spec.rb 1970-01-01 01:00:00.000000000 +0100 @@ -61,7 +61,7 @@ Config.load({}) end - context "loading '/dev/null' as spec tests do" do + context "loading '/dev/null' as spec tests do", :unless => Hiera::Util.microsoft_windows? do before :each do # Simulate the behavior of YAML.load_file('/dev/null') in MRI 1.9.3p194 Config.stubs(:yaml_load_file). @@ -69,7 +69,7 @@ end it "is not exceptional behavior" do - expect { Config.load('/dev/null') }.to_not raise_error + Config.load('/dev/null') end end end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/spec/unit/filecache_spec.rb new/spec/unit/filecache_spec.rb --- old/spec/unit/filecache_spec.rb 1970-01-01 01:00:00.000000000 +0100 +++ new/spec/unit/filecache_spec.rb 1970-01-01 01:00:00.000000000 +0100 @@ -1,62 +1,141 @@ require 'spec_helper' +require 'tmpdir' class Hiera describe Filecache do before do - File.stubs(:exist?).returns(true) @cache = Filecache.new end + def write_file(file, contents) + File.open(file, 'w') do |f| + f.write(contents) + end + end + describe "#read" do - it "should cache and read data" do - File.expects(:read).with("/nonexisting").returns("text") - @cache.expects(:path_metadata).returns(File.stat(__FILE__)).once - @cache.expects(:stale?).once.returns(false) + it "reads data from a file" do + Dir.mktmpdir do |dir| + file = File.join(dir, "testing") + write_file(file, "my data") + + @cache.read(file).should == "my data" + end + end - @cache.read("/nonexisting").should == "text" - @cache.read("/nonexisting").should == "text" + it "rereads data when the file changes" do + Dir.mktmpdir do |dir| + file = File.join(dir, "testing") + write_file(file, "my data") + @cache.read(file).should == "my data" + + write_file(file, "changed data") + @cache.read(file).should == "changed data" + end end - it "should support validating return types and setting defaults" do - File.expects(:read).with("/nonexisting").returns('{"rspec":1}') + it "uses the provided default when the type does not match the expected type" do + Hiera.expects(:debug).with(regexp_matches(/String.*not.*Hash, setting defaults/)) + Dir.mktmpdir do |dir| + file = File.join(dir, "testing") + write_file(file, "my data") + data = @cache.read(file, Hash, { :testing => "hash" }) do |data| + "a string" + end - @cache.expects(:path_metadata).returns(File.stat(__FILE__)) + data.should == { :testing => "hash" } + end + end - Hiera.expects(:debug).with(regexp_matches(/is not a Hash, setting defaults/)) + it "traps any errors from the block and uses the default value" do + Hiera.expects(:debug).with(regexp_matches(/Reading data.*failed:.*testing error/)) + Dir.mktmpdir do |dir| + file = File.join(dir, "testing") + write_file(file, "my data") + data = @cache.read(file, Hash, { :testing => "hash" }) do |data| + raise ArgumentError, "testing error" + end - # return bogus data on purpose, triggers setting defaults - data = @cache.read("/nonexisting", Hash, {"rspec" => 1}) do |data| - nil + data.should == { :testing => "hash" } end + end - data.should == {"rspec" => 1} + it "raises an error when there is no default given and there is a problem" do + Dir.mktmpdir do |dir| + file = File.join(dir, "testing") + write_file(file, "my data") + + expect do + @cache.read(file, Hash) do |data| + raise ArgumentError, "testing error" + end + end.to raise_error(ArgumentError, "testing error") + end end end - describe "#stale?" do - it "should return false when the file has not changed" do - stat = File.stat(__FILE__) - - @cache.stubs(:path_metadata).returns(stat) - @cache.stale?("/nonexisting").should == true - @cache.stale?("/nonexisting").should == false + describe "#read_file" do + it "reads data from a file" do + Dir.mktmpdir do |dir| + file = File.join(dir, "testing") + write_file(file, "my data") + + @cache.read_file(file).should == "my data" + end end - it "should update and return true when the file changed" do - @cache.expects(:path_metadata).returns({:inode => 1, :mtime => Time.now, :size => 1}) - @cache.stale?("/nonexisting").should == true - @cache.expects(:path_metadata).returns({:inode => 2, :mtime => Time.now, :size => 1}) - @cache.stale?("/nonexisting").should == true + it "rereads data when the file changes" do + Dir.mktmpdir do |dir| + file = File.join(dir, "testing") + write_file(file, "my data") + @cache.read_file(file).should == "my data" + + write_file(file, "changed data") + @cache.read_file(file).should == "changed data" + end + end + + it "errors when the type does not match the expected type" do + Dir.mktmpdir do |dir| + file = File.join(dir, "testing") + write_file(file, "my data") + + expect do + @cache.read_file(file, Hash) do |data| + "a string" + end + end.to raise_error(TypeError) + end end - end - describe "#path_metadata" do - it "should return the right data" do - stat = File.stat(__FILE__) + it "converts the read data using the block" do + Dir.mktmpdir do |dir| + file = File.join(dir, "testing") + write_file(file, "my data") + + @cache.read_file(file, Hash) do |data| + { :data => data } + end.should == { :data => "my data" } + end + end - File.expects(:stat).with("/nonexisting").returns(stat) + it "errors when the file does not exist" do + expect do + @cache.read_file("/notexist") + end.to raise_error(Errno::ENOENT) + end - @cache.path_metadata("/nonexisting").should == {:inode => stat.ino, :mtime => stat.mtime, :size => stat.size} + it "propogates any errors from the block" do + Dir.mktmpdir do |dir| + file = File.join(dir, "testing") + write_file(file, "my data") + + expect do + @cache.read_file(file) do |data| + raise ArgumentError, "testing error" + end + end.to raise_error(ArgumentError, "testing error") + end end end end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/spec/unit/version_spec.rb new/spec/unit/version_spec.rb --- old/spec/unit/version_spec.rb 1970-01-01 01:00:00.000000000 +0100 +++ new/spec/unit/version_spec.rb 1970-01-01 01:00:00.000000000 +0100 @@ -0,0 +1,44 @@ +require "spec_helper" +require "hiera/version" +require 'pathname' + +describe "Hiera.version Public API" do + subject() { Hiera } + + before :each do + Hiera.instance_eval do + if @hiera_version + @hiera_version = nil + end + end + end + + context "without a VERSION file" do + before :each do + subject.stubs(:read_version_file).returns(nil) + end + + it "is Hiera::VERSION" do + subject.version.should == Hiera::VERSION + end + it "respects the version= setter" do + subject.version = '1.2.3' + subject.version.should == '1.2.3' + end + end + + context "with a VERSION file" do + it "is the content of the file" do + subject.expects(:read_version_file).with() do |path| + pathname = Pathname.new(path) + pathname.basename.to_s == "VERSION" + end.returns('1.2.1-9-g9fda440') + + subject.version.should == '1.2.1-9-g9fda440' + end + it "respects the version= setter" do + subject.version = '1.2.3' + subject.version.should == '1.2.3' + end + end +end -- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
