Jira (HI-118) There's no way to set resolution_type when using data bindings
Title: Message Title Rob Nelson commented on HI-118 Re: There's no way to set resolution_type when using data bindings Henrik Lindberg I didn't find any tests, but I documented my testing methods and attached both code and document PRs to this ticket. They're ready for review on this week's triage call, I should be able to attend it. Here's hoping to making Puppet 4.3.0 Add Comment This message was sent by Atlassian JIRA (v6.4.11#64026-sha1:78f6ec4) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (HI-118) There's no way to set resolution_type when using data bindings
Title: Message Title Henrik Lindberg commented on HI-118 Re: There's no way to set resolution_type when using data bindings Rob Nelson Have not looked at where similar tests are located, I would imagine they are related to data binding. It may also be that there are very few tests since it has always been troublesome with the circular dependencies between hiera and puppet; instead there is probably just unit tests for the respective features in the respective project. We can help sort out testing and documentation. We are however close to feature complete cutoff for Puppet 4.3.0, which means that we are swamped with work in need of final touches, so I cannot promise that we will have time to take this on, but we will do our best. Since the change can be done without a new hiera release being required, there is a bigger chance of getting this in. Add Comment This message was sent by Atlassian JIRA (v6.4.11#64026-sha1:78f6ec4) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (HI-118) There's no way to set resolution_type when using data bindings
Title: Message Title Rob Nelson commented on HI-118 Re: There's no way to set resolution_type when using data bindings I'm almost ready to submit the PR on puppet (will need to do puppet-docs as well) but I can't find any existing related tests. I'm modifying the function at https://github.com/puppetlabs/puppet/blob/master/lib/puppet/resource/type.rb#L564 and would expect those tests at https://github.com/puppetlabs/puppet/blob/master/spec/unit/resource/type_spec.rb. I'm not sure how to craft a new test to call lookup_external_default_for() with a mock up. I'll submit the PR without tests unless someone can point me in the right direction. Add Comment This message was sent by Atlassian JIRA (v6.4.11#64026-sha1:78f6ec4) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (HI-118) There's no way to set resolution_type when using data bindings
Title: Message Title John Bollinger commented on HI-118 Re: There's no way to set resolution_type when using data bindings The proposed solution seems a win to me, but it also seems to be missing an opportunity to be an even bigger win. Going back to my May, 2014 comments: Bottom line: hiera_hash() and hiera_array() should both be deprecated in favor of a means to express the intended retrieval type directly in the data. The hiera() function, whether invoked from the DSL or internally by Puppet, should use that means to retrieve the data correctly. To put it another way, the problem stated in the title of this issue is only one aspect of a deeper problem, which I described in those earlier comments. Those comments seemed to be acknowledged and well received at the time. I really hope the whole problem can be solved, especially since a partial solution now greatly reduces the likelihood that a complete solution will ever be implemented. Add Comment This message was sent by Atlassian JIRA (v6.4.11#64026-sha1:78f6ec4) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (HI-118) There's no way to set resolution_type when using data bindings
Title: Message Title Henrik Lindberg commented on HI-118 Re: There's no way to set resolution_type when using data bindings John Bollinger Completely agree with you. The current proposal is really a stop-gap solution for automatic data binding only. It would be much better if the data composition was completely done in the data and that it does not require the user to know how to look something up. The problem with implementing this is that we have not come up with a good solution that would be both performant and backwards compatible; the hiera data formats pretty much blocks all attempts to add meaning to any combinations of keys and values. For that reason, we decided that the first implementation of "Data in Modules" (which is really the beginning of a refactored and improved Hiera) should be 100% data file compatible. Going forward, I think we should introduce new backends that when used allows richer features in the data files themselves; awareness of data types, ability to alias/interpolate/combine/transform values other than strings; which includes merging of arrays and hashes, to mention a few. If you recall ARM-8, this was very much a goal of that proposal. It turned out to be too complex and was rejected as the mechanism for data binding. There are however many concepts from that ARM that we can use and make available in simpler form expressed in hiera data. As the proposal at hand now provides a much sought after feature I think it is of value to get it into a Hiera 3 release even if we do not yet have a design for the desired features, and that it means having to make some changes to data files when such a solution is available. I think that is more favorable than waiting for the additional desired features to be designed. Add Comment This message was sent by Atlassian JIRA (v6.4.11#64026-sha1:78f6ec4) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to
Jira (HI-118) There's no way to set resolution_type when using data bindings
Title: Message Title Rob Nelson commented on HI-118 Re: There's no way to set resolution_type when using data bindings John: This would only be for automatic parameter lookup according to the merge strategy defined at https://github.com/puppetlabs/puppet/blob/d2199af906fa5ca19ec0c37c82794934ea4fb4b5/lib/puppet/functions/lookup.rb#L58-L59 This would work for arrays as well as hashes. The lookupoptions are done as a first-fit lookup (standard). Since the value can be set at any level of the hierarchy this lets the user define at whatever level makes sense, without worrying about conflicts or recursion. It's possible in the future to set up lookupoptions::* for the class, but that won't be in this first revision. I'll see if I can get some docs and tests ready this weekend. Add Comment This message was sent by Atlassian JIRA (v6.4.11#64026-sha1:78f6ec4) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (HI-118) There's no way to set resolution_type when using data bindings
Title: Message Title John Bollinger commented on HI-118 Re: There's no way to set resolution_type when using data bindings +1 for lookupoptions. This seems to be a viable way to define the lookup type on a per-item basis. Questions: Is it intended that a merge strategy being defined for a given key indicates that ordinary lookups for that key should employ merging (of the specified flavor)? The alternative would be that only hash-merge lookups that happen to be performed for the key, as requested by some other means, would employ the specified strategy. The details presented in the proposal seem specific to hash-valued items. Supposing that the answer to my previous question is "yes", is it intended or desirable that array lookup type can be specified this way as well (i.e. as if hiera_array() were used)? I think the proposal says that the lookupoptions::* keys are intended themselves to be looked up in the data hierarchy. What type of lookup is used for them? Does the facility they provide apply recursively? Add Comment This message was sent by Atlassian JIRA (v6.4.11#64026-sha1:78f6ec4) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (HI-118) There's no way to set resolution_type when using data bindings
Title: Message Title Henrik Lindberg commented on HI-118 Re: There's no way to set resolution_type when using data bindings The options should be given as for the lookup function (it covers the functionality of hiera, hiera_array and hiera_hash and works with both hiera as well as "data in modules" (a.k.a data providers). The puppet setting should not be specific to "hiera" - it is a data-binder feature. The lookup options for the key only applies to automatic data binding. For other use cases (i.e. directly calling lookup the user can manually look up the lookup options for the key. For manual calls to the hiera_xxx functions, the user also needs to pick a specific hiera_xxx call based on the options as well as transforming the options to adapt to the hiera_xxx functions limited set of features. Add Comment This message was sent by Atlassian JIRA (v6.4.11#64026-sha1:78f6ec4) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (HI-118) There's no way to set resolution_type when using data bindings
Title: Message Title Rob Nelson commented on HI-118 Re: There's no way to set resolution_type when using data bindings At the contributor's summit last week, Henrik, Felix, Vanessa and myself hashed out a potential solution. Our proposal requires two pieces. In puppet.conf's master section, hiera_advanced_parameter_bindings must be set to true. [master] hiera_advanced_parameter bindings = true If a key of lookupoptions:: is found and it contains the hash key 'merge' with a valid value (typically 'deep'), the APL merge strategy is set to the specified value. This key can be found anywhere but it is suggested to be placed in a more specific location, to give per-tier control over the merge strategy (i.e. do not set it in global/common, set it in a clientcert or datacenter/network/etc. type hierarchy level). It would look like this: lookupoptions::mysql::server::overrideoptions: merge: deep Then, any keys matching would be found during APL and merged using the specified behavior. You can see the few changes required to make the APL lookups work at https://github.com/ffrank/puppet/commit/9ba281986f88cb9d889fa02a650779ef92d0b6eb. If this proposal is accepted, the puppet.conf setting and tests can be generated and docs fleshed out, but I'd like to know if this proposal is sufficient before fleshing out the changes and tweaking debug messages, etc. Thanks to Henrik, Vanessa, and Felix for their assistance during the summit!
Jira (HI-118) There's no way to set resolution_type when using data bindings
Title: Message Title garrett honeycutt commented on HI-118 Re: There's no way to set resolution_type when using data bindings +1 I like this approach because it is a clear pattern that can be expressed entirely in Hiera. With regards to my example, it moves the logic from being implemented in the module to a standard interface in Hiera, saving me a ton of code and spec tests Add Comment This message was sent by Atlassian JIRA (v6.4.11#64026-sha1:78f6ec4) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (HI-118) There's no way to set resolution_type when using data bindings
Title: Message Title
Steve Traylen commented on HI-118
Re: There's no way to set resolution_type when using data bindings
Thisi is not possible since hiera_array cannot be used as a lookup functions. However if it were possible the following might work so long as it's the winning setting under current data binding.
---
mymodule::settings:
- value1
- value2
- {hiera_array('mymodule::settings')}
clearly some potential recursion problems but.
Add Comment
This message was sent by Atlassian JIRA (v6.3.7#6337-sha1:2ed701e)
Jira (HI-118) There's no way to set resolution_type when using data bindings
Title: Message Title Robert commented on an issue Re: There's no way to set resolution_type when using data bindings It would be nice if there were some way to define this merge behavior, as it would seriously increase the flexibility and general awesomeness of hiera. Add Comment Hiera / HI-118 There's no way to set resolution_type when using data bindings Because it's the responsibility of the caller to set `resolution_type` in hiera, and there is no explicit caller when using data bindings, there's no way to tell hiera to use the array (append) or hash (merge) resolutions. This is a pretty bad flaw as it prevents sophisticated lookups like deep merges (or, well, any merges at all) when using the fancy dat... This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit
Jira (HI-118) There's no way to set resolution_type when using data bindings
Title: Message Title John Bollinger commented on an issue Re: There's no way to set resolution_type when using data bindings When I thought a bit about this because of a question on puppet-users, I realized that it is not an issue that can be solved by a change to Puppet DSL or to the way Puppet uses Hiera internally. Whether a priority lookup, a hash-merge lookup, or an array lookup is wanted in a particular data-binding situation is not (or at least should not be) a function of the class. Rather, it is an aspect of the data themselves. That is, even if there were a way to express in a class definition that a particular parameter should be bound via a hash-merge lookup (for example), that would just turn the problem around on people who want to use a priority lookup for that parameter. And, wow, I just saw the broader implication: the same argument applies to any use of hiera_array() or hiera_hash(). The intended meaning of the data is part of the data definition, and looking it up via a different function than intended changes that meaning. Hiera (the YAML back end) does not have a mechanism to express this aspect of the data, so the data consumer is left to guess which of three functions is the appropriate one to use for retrieval, without even a means to check after the fact whether they guessed right. Bottom line: hiera_hash() and hiera_array() should both be deprecated in favor of a means to express the intended retrieval type directly in the data. The hiera() function, whether invoked from the DSL or internally by Puppet, should use that means to retrieve the data correctly. Add Comment Hiera / HI-118 There's no way to set resolution_type when using data bindings Because it's the responsibility of the caller to set `resolution_type` in hiera, and there is no explicit caller when using data bindings, there's no way to tell hiera to use the array (append) or hash
Jira (HI-118) There's no way to set resolution_type when using data bindings
Title: Message Title garrett honeycutt commented on an issue Re: There's no way to set resolution_type when using data bindings +1 Currently I am attempting to account for the use of merge lookups in puppet code, which is not very nice. https://github.com/ghoneycutt/puppet-module-ssh/blob/v3.6.0/manifests/init.pp#L280-286 Add Comment Hiera / HI-118 There's no way to set resolution_type when using data bindings Because it's the responsibility of the caller to set `resolution_type` in hiera, and there is no explicit caller when using data bindings, there's no way to tell hiera to use the array (append) or hash (merge) resolutions. This is a pretty bad flaw as it prevents sophisticated lookups like deep merges (or, well, any merges at all) when using the fancy dat... This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at
Jira (HI-118) There's no way to set resolution_type when using data bindings
Title: Message Title Charlie Sharpsteen updated an issue Hiera / HI-118 There's no way to set resolution_type when using data bindings Change By: Charlie Sharpsteen Issue Type: Bug NewFeature Add Comment This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/groups/opt_out.
Jira (HI-118) There's no way to set resolution_type when using data bindings
Title: Message Title Charlie Sharpsteen commented on an issue Re: There's no way to set resolution_type when using data bindings This one is tricky because it might require a change in how parameterized classes are declared. Eric Sorenson looks like this needs a decision Add Comment Hiera / HI-118 There's no way to set resolution_type when using data bindings Because it's the responsibility of the caller to set `resolution_type` in hiera, and there is no explicit caller when using data bindings, there's no way to tell hiera to use the array (append) or hash (merge) resolutions. This is a pretty bad flaw as it prevents sophisticated lookups like deep merges (or, well, any merges at all) when using the fancy dat... This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at
Jira (HI-118) There's no way to set resolution_type when using data bindings
Title: Message Title Eric Sorenson commented on an issue Re: There's no way to set resolution_type when using data bindings No, what it needs is a some technical design work. Add Comment Hiera / HI-118 There's no way to set resolution_type when using data bindings Because it's the responsibility of the caller to set `resolution_type` in hiera, and there is no explicit caller when using data bindings, there's no way to tell hiera to use the array (append) or hash (merge) resolutions. This is a pretty bad flaw as it prevents sophisticated lookups like deep merges (or, well, any merges at all) when using the fancy dat... This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/groups/opt_out.
Jira (HI-118) There's no way to set resolution_type when using data bindings
Title: Message Title Eric Sorenson updated an issue Hiera / HI-118 There's no way to set resolution_type when using data bindings Change By: Eric Sorenson Assignee: EricSorenson AndrewParker Add Comment This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/groups/opt_out.
Jira (HI-118) There's no way to set resolution_type when using data bindings
Title: Message Title Eric Sorenson commented on an issue Re: There's no way to set resolution_type when using data bindings Andy I've put this in Designing and assigned it to you for assessment or delegation Add Comment Hiera / HI-118 There's no way to set resolution_type when using data bindings Because it's the responsibility of the caller to set `resolution_type` in hiera, and there is no explicit caller when using data bindings, there's no way to tell hiera to use the array (append) or hash (merge) resolutions. This is a pretty bad flaw as it prevents sophisticated lookups like deep merges (or, well, any merges at all) when using the fancy dat... This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/groups/opt_out.
Jira (HI-118) There's no way to set resolution_type when using data bindings
Title: Message Title redmine.exporter created an issue Hiera / HI-118 There's no way to set resolution_type when using data bindings Issue Type: Bug Assignee: Eric Sorenson Created: 17/Dec/13 5:43 PM Labels: redmine Priority: Normal Reporter: redmine.exporter Because it's the responsibility of the caller to set `resolution_type` in hiera, and there is no explicit caller when using data bindings, there's no way to tell hiera to use the array (append) or hash (merge) resolutions. This is a pretty bad flaw as it prevents sophisticated lookups like deep merges (or, well, any merges at all) when using the fancy data bindings. Add Comment
