Jira (HI-6) Hiera needs to validate the structure of its config file
Title: Message Title Eric Thompson commented on HI-6 Re: Hiera needs to validate the structure of its config file we haven't tested for edge cases in config structure specifically. we could probably get more bang in unit tests for stuff like that. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (HI-6) Hiera needs to validate the structure of its config file
Title: Message Title Charlie Sharpsteen commented on HI-6 Re: Hiera needs to validate the structure of its config file If Hiera 5 validates the structure of the yaml file, then I think that's definitely good enough. The problem here was that Hiera would silently accept a configuration with an incorrect structure for core values like datadir — and then mysterious errors of the sort that take a long time to trace down would start occurring. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (HI-6) Hiera needs to validate the structure of its config file
Title: Message Title Henrik Lindberg commented on HI-6 Re: Hiera needs to validate the structure of its config file Eric Sorenson Hiera 5 actually reads hiera 3 format yaml and checks the structure - all brand new code. Basically Hiera 5 takes over a lot of the upper levels of the hiera 3 logic. If there are flaws in Puppet 4.9.0/hiera 5 there should be PUP tickets logged for that. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (HI-6) Hiera needs to validate the structure of its config file
Title: Message Title Eric Sorenson commented on HI-6 Re: Hiera needs to validate the structure of its config file I'm inclined to close this as won't-fix - as long as hiera 5 has better error checking for these kinds of config file typos. I acknowledge it's a problem but it seems not worth the time to dig in and validate each of the kinds of yaml errors that could arise. Eric Thompson have you tried this kind of file validation as part of automated or manual hiera.yaml testing? If so great, if not let's spin this into related PUP tickets. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (HI-6) Hiera needs to validate the structure of its config file
Title: Message Title Henrik Lindberg commented on HI-6 Re: Hiera needs to validate the structure of its config file There are providers that do not use datadir at all - would they fail? Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (HI-6) Hiera needs to validate the structure of its config file
Title: Message Title Charlie Sharpsteen updated an issue Hiera / HI-6 Hiera needs to validate the structure of its config file Change By: Charlie Sharpsteen Affects Version/s: HI 3.2.1 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (HI-6) Hiera needs to validate the structure of its config file
Title: Message Title Charlie Sharpsteen updated an issue Hiera / HI-6 Hiera needs to validate the structure of its config file Change By: Charlie Sharpsteen Affects Version/s: HI 1.3.4 Affects Version/s: HI 2.0.0 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (HI-6) Hiera needs to validate the structure of its config file
Title: Message Title Eric Sorenson updated an issue Hiera / HI-6 Hiera needs to validate the structure of its config file Change By: Eric Sorenson Assignee: EricSorenson Add Comment This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/groups/opt_out.
Jira (HI-6) Hiera needs to validate the structure of its config file
Title: Message Title Eric Sorenson updated an issue Hiera / HI-6 Hiera needs to validate the structure of its config file Change By: Eric Sorenson Assignee: CharlieSharpsteen Add Comment This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/groups/opt_out.
Jira (HI-6) Hiera needs to validate the structure of its config file
Title: Message Title
Charlie Sharpsteen updated an issue
Hiera / HI-6
Hiera needs to validate the structure of its config file
Change By:
Charlie Sharpsteen
Hieradoeshavea[validatemethodinconfig.rb ][ |https://github.com/puppetlabs/ hiera /blob/1.2.1/lib/hiera/config.rb#L51 - validator L62 ]thatisappliedtothedatastructureloadedfrom`hiera.yaml`.Unfortunately,thisvalidatorcurrentlyonlytrapserrorsarisingfrommissingthe`deep_merge`gem. Hieraassumesthattheconfigfilewillfollowacertainstructureandwillcrashormisbehaveinstrangewaysifthisstructureisnotpresent.Forexample,[the:backends:and:hierarchy:keyscanbesettoarraysorsinglestrings ][ |http://docs.puppetlabs.com/ hiera /1/configuring.html#example -config -file ].However,the`:datadir`keywillmisbehaveinweirdwaysifsettoanythingotherthanastring: {code} #Thedocumentedstructureforthissettingisasinglestring :yaml: :datadir:/path/to/some/dir/%{::environment} #Bycoincidence,usinganarraywillwork :yaml: :datadir: -/path/to/some/dir #But...variableinterpolationwillmysteriouslyfail :yaml: :datadir: -/path/to/some/dir/%{::environment} [hiera-validator]:https://github.com/puppetlabs/hiera/blob/1.2.1/lib/hiera/config.rb#L51-L62[hiera-config]:http://docs.puppetlabs.com/hiera/1/configuring.html#example-config-file {code}
Add Comment
This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede)
--
You received this message because you are subscribed to the Google Groups Puppet Bugs group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-bugs@googlegroups.com.
Visit this group at
Jira (HI-6) Hiera needs to validate the structure of its config file
Title: Message Title Charlie Sharpsteen commented on an issue Re: Hiera needs to validate the structure of its config file Level of impact (not sure whether to make this public, so going with private): YAML is prone to very subtle errors that can completely change the structure of the loaded config. This is also compounded by the whitespace-sensitive nature of the format. In this case, it took two support engineers two days of comparing notes and test environments to notice a misplaced hyphen as Hiera wasn't visibly complaining about loading its config — it was just failing to provide a customer with correct data for their manifests. Much debugging time was spent probing the data binding system in Puppet for problems. In the case of the `datadir` setting, some very cursory structural validation would have saved the day: @config[:backends].each do |backend| raise Error unless @config[backend][:datadir].is_a? String end Add Comment Hiera / HI-6 Hiera needs to validate the structure of its config file Hiera does have a [validate method in config.rb|https://github.com/puppetlabs/hiera/blob/1.2.1/lib/hiera/config.rb#L51-L62] that is applied to the data structure loaded from `hiera.yaml`. Unfortunately, this validator currently only traps errors arising from missing the `deep_merge` gem. Hiera assumes that the config file will follow a certain structur...
