Stack,
On Thu, Sep 17, 2015 at 12:41 PM, Stack Kororā <[email protected]> wrote: > Greetings, > > I am having an issue with Hiera again. Bangin my head on it for a few hours > and not getting anywhere. Since this group was able to help last time, I > have my fingers crossed you can help me again. :-D > > I am trying to use JSON with Hiera this time. > > First, fully updated dev box. > $ cat /etc/redhat-release > Scientific Linux release 6.7 (Carbon) > > $ sudo yum update > Loaded plugins: refresh-packagekit, security > Setting up Update Process > No Packages marked for Update > > $ rpm -qa | egrep '(puppet|fact|hiera|ruby)' > facter-2.4.4-1.el6.x86_64 > puppet-3.8.2-1.el6.noarch > hiera-1.3.4-1.el6.noarch > puppet-server-3.8.2-1.el6.noarch > ruby-libs-1.8.7.374-4.el6_6.x86_64 > ruby-rdoc-1.8.7.374-4.el6_6.x86_64 > ruby-shadow-2.2.0-2.el6.x86_64 > ruby-1.8.7.374-4.el6_6.x86_64 > rubygems-1.3.7-5.el6.noarch > ruby-augeas-0.4.1-3.el6.x86_64 > rubygem-deep_merge-1.0.0-2.el6.noarch > libselinux-ruby-2.0.94-5.8.el6.x86_64 > rubygem-json-1.5.5-3.el6.x86_64 > ruby-irb-1.8.7.374-4.el6_6.x86_64 > > > Let's use the puppet documentation to get started, shall we? > https://docs.puppetlabs.com/hiera/1/data_sources.html > > $ cat /etc/puppet/hiera.yaml > --- > :backends: > - json > :hierarchy: > - common > :merge_behavior: "deeper" > :json: > :datadir: /etc/puppet/hieradata > > BTW: the doc's actual JSON example is broken as it is missing the [ ] around > it. I included it. below and if you don't you fail the parser and Hiera just > ignores everything instead of giving an error. > Adding [ ] around your json is what is causing the problem you're seeing. It turns the data you are writing into an Array instead of a Hash. Hiera is expecting it to be a hash, as the error message indicates. The error message here may not be very helpful though, because the data isn't a string, but is an Array. What parse error do you get without the [ ]? I was able to get a correct return from hiera by removing the [ ]. > $ cat /etc/puppet/hieradata/common.json > [ > { > "apache-packages" : [ > "apache2", > "apache2-common", > "apache2-utils" > ], > > "hosts_entry" : "sandbox.%{fqdn}", > > "sshd_settings" : { > "root_allowed" : "no", > "password_allowed" : "no" > } > } > ] > > Verify valid JSON > $ cat /etc/puppet/hieradata/common.json | jq '.[0]' > { > "sshd_settings": { > "password_allowed": "no", > "root_allowed": "no" > }, > "hosts_entry": "sandbox.%{fqdn}", > "apache-packages": [ > "apache2", > "apache2-common", > "apache2-utils" > ] > } > > Awesome! Lets validate in Hiera: > $ hiera --version > 1.3.4 > $ hiera -c /etc/puppet/hiera.yaml hosts_entry > /usr/lib/ruby/site_ruby/1.8/hiera/filecache.rb:56:in `read_file': Data > retrieved from /etc/puppet/hieradata/common.json is String not Hash > (TypeError) > from /usr/lib/ruby/site_ruby/1.8/hiera/backend/json_backend.rb:24:in > `lookup' > from /usr/lib/ruby/site_ruby/1.8/hiera/backend.rb:76:in `datasources' > from /usr/lib/ruby/site_ruby/1.8/hiera/backend.rb:74:in `map' > from /usr/lib/ruby/site_ruby/1.8/hiera/backend.rb:74:in `datasources' > from /usr/lib/ruby/site_ruby/1.8/hiera/backend/json_backend.rb:17:in > `lookup' > from /usr/lib/ruby/site_ruby/1.8/hiera/backend.rb:206:in `lookup' > from /usr/lib/ruby/site_ruby/1.8/hiera/backend.rb:203:in `each' > from /usr/lib/ruby/site_ruby/1.8/hiera/backend.rb:203:in `lookup' > from /usr/lib/ruby/site_ruby/1.8/hiera.rb:60:in `lookup' > from /usr/bin/hiera:225 > > > > And here is where things went wrong. The worst part is that it is such a > generic error I can't find anything of real value in my online searching. > Even worse, the only directly relevant hit I found basically said to stop > using JSON and use YAML instead. Not really a solution is it? > > But whatever. It is probably good to verify that YAML is at least working > with Heira in the dev environment. Using the puppet documentation again for > guidance. > $ mv /etc/puppet/hieradata/common.json /etc/puppet/hieradata/common.yaml > $ vim /etc/puppet/hieradata/common.yaml > $ cat /etc/puppet/hieradata/common.yaml > --- > # array > apache-packages: > - apache2 > - apache2-common > - apache2-utils > > # string > apache-service: apache2 > > # interpolated facter variable > hosts_entry: "sandbox.%{fqdn}" > > # hash > sshd_settings: > root_allowed: "no" > password_allowed: "yes" > > # alternate hash notation > sshd_settings: {root_allowed: "no", password_allowed: "yes"} > > # to return "true" or "false" > sshd_settings: {root_allowed: no, password_allowed: yes} > > > $ vim /etc/puppet/hiera.yaml > $ cat /etc/puppet/hiera.yaml > --- > :backends: > - yaml > :hierarchy: > - common > :merge_behavior: "deeper" > :yaml: > :datadir: /etc/puppet/hieradata > > > $ hiera -c /etc/puppet/hiera.yaml hosts_entry > sandbox. > > Awesome! So YAML works. > > Back to the JSON version.... > > $ hiera -c /etc/puppet/hiera.yaml hosts_entry > /usr/lib/ruby/site_ruby/1.8/hiera/filecache.rb:56:in `read_file': Data > retrieved from /etc/puppet/hieradata/common.json is String not Hash > (TypeError) > from /usr/lib/ruby/site_ruby/1.8/hiera/backend/json_backend.rb:24:in > `lookup' > from /usr/lib/ruby/site_ruby/1.8/hiera/backend.rb:76:in `datasources' > from /usr/lib/ruby/site_ruby/1.8/hiera/backend.rb:74:in `map' > from /usr/lib/ruby/site_ruby/1.8/hiera/backend.rb:74:in `datasources' > from /usr/lib/ruby/site_ruby/1.8/hiera/backend/json_backend.rb:17:in > `lookup' > from /usr/lib/ruby/site_ruby/1.8/hiera/backend.rb:206:in `lookup' > from /usr/lib/ruby/site_ruby/1.8/hiera/backend.rb:203:in `each' > from /usr/lib/ruby/site_ruby/1.8/hiera/backend.rb:203:in `lookup' > from /usr/lib/ruby/site_ruby/1.8/hiera.rb:60:in `lookup' > from /usr/bin/hiera:225 > > > Dah! > > No matter how simple I make it or what tweaks I do to the common.json file, > I always get that exact error. If I run puppet agent, I get a very similar > error too! > > "So just use YAML like the other post said" > 1) That doesn't fix JSON and it avoids the problem ("Hey doc, my legs are > broken!" "Have you tried walking on your hands instead? Why not just do > that?" :-P) > > 2) The fancy new tool that my security team is using to manage and monitor > system wide variables reads/writes JSON but doesn't do YAML. Meaning we > either have to keep doing the manual checks we are doing, or stick a parser > in between. Why? Puppet and Heira are supposed to work with JSON, it > read/writes JSON, this should theoretically just work. > > Does anyone know why hiera isn't working with JSON? I really feel like this > is probably a simple solution that I am just not seeing. I am hoping someone > else might be able to recognize the answer. > > Thanks! > ~Stack~ > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/puppet-users/2703e86d-f1e8-40a3-9d0c-24fd85bbbfe4%40googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- Matthaus Owens Puppet Labs PuppetConf 2015 is coming to Portland, Oregon! Join us October 5-9. Register now to take advantage of the Early Bird discount —save $249! -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CACD%3DwAde7ZJS5ECkFEZaeMG1M-XukW5HKtJnCSRo8-9Sw3--AA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
