On Monday, October 14, 2013 9:19:10 AM UTC+2, Erik Dalén wrote:
>
>
>
>
> On 13 October 2013 12:40, Alessandro Franceschi <[email protected]<javascript:>
> > wrote:
>
>> Thanks for the update Eric, very useful to understand the ongoing works
>> on data in modules.
>>
>> On Friday, October 11, 2013 9:01:19 PM UTC+2, Dan Bode wrote:
>>
>>>
>>>
>>>
>>> On Fri, Oct 11, 2013 at 11:09 AM, Eric Sorenson <
>>> [email protected]> wrote:
>>>
>>>>
>>>> Thanks to everyone who kicked the tires on the experimental data in
>>>> modules feature included in Puppet 3.3.0. We got a lot of feedback, some
>>>> cool proof-of-concept modules, and a definitive conclusion to the
>>>> experiment.
>>>>
>>>> The idea of including a module-specific hiera backend is centered
>>>> around one primary use case: replacing the 'params class pattern', a
>>>> common
>>>> idiom in Puppet modules that's described in the [Using Parameterized
>>>> Classes][param-classes] guide. The problem that most testers ran into
>>>> though is that for non-trivial modules they ended up having to
>>>> re-implement
>>>> the Puppet DSL logic encoded in their params.pp in convoluted, non-obvious
>>>> ways. The solutions to this led to more contortions until we'd ended up
>>>> with the ability to execute parser functions in the right-hand-side of a
>>>> yaml value. So something which started out trying to help separate data
>>>> from code ended up putting code back into data!
>>>>
>>>> Additionally, even after multiple attempts to simplify the surface area
>>>> and user experience with the bindings system (described in ARM-9) that
>>>> underlay the data-in-modules implementation, users still found its
>>>> complexity daunting. There are some important bits of scaffolding (like an
>>>> actual type system for Puppet!) that will prove valuable as more of the
>>>> future parser and evaluator work that Henrik is building makes its way
>>>> into
>>>> the product, but in the final analysis the data in modules feature was the
>>>> wrong vehicle to introduce them.
>>>>
>>>> Refocusing on the problems users were trying to solve (and here I have
>>>> to give shout-outs to Ashley Penney for his [puppetlabs-ntp][] branch and
>>>> the dynamic duo of Spencer Krug/William van Hevelingen for their
>>>> [startrek][] module) and the problems with the 'params' pattern lent some
>>>> clarity. We've gotten into a situation of disparity with regard to hiera
>>>> and data bindings, because data bindings enable module _users_ to use
>>>> their
>>>> site-wide hiera data but don't provide moduel _authors_ the same
>>>> affordance. But rather than introduce additional complexity, we can close
>>>> the gap for existing code patterns.
>>>>
>>>> So the proposed solution at this point is:
>>>> - enable an implicit data-binding lookup against the hiera-puppet
>>>> backend for a value of 'classname::variable' in the file
>>>> 'modules/classname/manifests/**params.pp', which simplifies class
>>>> definition and provides consistency with other hiera backends. As a module
>>>> author, you'd still leave your logic for variables in params.pp, but
>>>> they'd
>>>> be implicitly looked up via data bindings as the class is declared, after
>>>> consulting site-wide hiera.
>>>>
>>>
>>> +1
>>>
>>> Really happy to see this solved in a way that will not lead to complex
>>> migrations to Puppet 4.
>>>
>>> Although, to play devil's advocate, two concerns:
>>>
>>> the special nature of params as a namespace suffix:
>>> - how do users know not to use this namespace for anything else?
>>> - What if user declares resources in params? Does this fail? Do they
>>> always get realized when anything else from that namespace is applied?
>>>
>>> the magic mapping from <x>::parameter <x>::params::parameter may be
>>> something hard to grok for new users who are not already familiar with the
>>> params pattern. This is probably solvable with documentation and --debug
>>> logging, but still worth noting.
>>>
>>
>> Yes worth noting, but as you said proper documentation might suffice.
>> Anyway +1 also for me on the lookup to params.pp (Dan this, + Puppet 3
>> data bindings, reminds me
>> https://github.com/example42/puppi/blob/master/lib/puppet/parser/functions/params_lookup.rb;-)
>>
>> I'd like to add to the complexity cases , in case you haven't still
>> sorted it out, the situations where some of the modules' params change
>> according to user provided values to other params.
>> For example look here:
>>
>> https://github.com/stdmod/puppet-elasticsearch/blob/master/manifests/init.pp#L124
>> the path of the configuration dir ($dir param here) change according to
>> the value of the $install param,
>> and in order to manage this the code in the linked line is necessary.
>> I wonder how such a case could be managed with data in modules (without
>> replicating a similar logic).
>>
>
> Isn't that possible to solve by allowing one hiera value to use another
> hiera value for interpolation?
> https://projects.puppetlabs.com/issues/21367
>
Didn't know of this function which actually looks interesting.
Is the hiera value usable in the same hierarchy structure ?
I mean, would it be possibile to have an hiera.yaml like:
---
version: 3
hierarchy:
- category: 'osfamily'
- category: 'operatingsystem'
- category: '^{install}'
- category: 'environment'
- category: 'common'
paths:
- 'is_virtual/${is_virtual}'
- 'common'
Is so, then it might actually work.
>
>
>>
>> al
>>
>>
>>>
>>>> - remove the user-facing '--binder' functionality
>>>> - fix known problems with the hiera-puppet lookups ([Redmine
>>>> 15746][15746], namely, but if there are others that are important to you
>>>> please speak up!)
>>>>
>>>> To show how this would work, I'll rework the ['smart parameter
>>>> defaults' example][param-classes] I linked above, with my commentary
>>>> behind
>>>> `##` comments:
>>>>
>>>> # /etc/puppet/modules/webserver/**manifests/params.pp
>>>>
>>>> class webserver::params { ## nothing changes here...
>>>> $packages = $operatingsystem ? {
>>>> /(?i-mx:ubuntu|debian)/ => 'apache2',
>>>> /(?i-mx:centos|fedora|redhat)**/ => 'httpd',
>>>> }
>>>> $vhost_dir = $operatingsystem ? {
>>>> /(?i-mx:ubuntu|debian)/ => '/etc/apache2/sites-enabled',
>>>> /(?i-mx:centos|fedora|redhat)**/ => '/etc/httpd/conf.d',
>>>> }
>>>> }
>>>>
>>>> # /etc/puppet/modules/webserver/**manifests/init.pp
>>>>
>>>> class webserver( ## inheritance is gone, and
>>>> $packages, ## data bindings look up the defaults
>>>> $vhost_dir ## as webserver::params::vhost_dir
>>>> ) {
>>>>
>>>> package { $packages: ensure => present }
>>>>
>>>> file { 'vhost_dir':
>>>> path => $vhost_dir,
>>>> ensure => directory,
>>>> mode => '0750',
>>>> owner => 'www-data',
>>>> group => 'root',
>>>> }
>>>> }
>>>>
>>>> # /etc/puppet/manifests/site.pp
>>>>
>>>> node default {
>>>> class { 'webserver': } ## no params needed, they're in hiera
>>>>
>>>> ## then in one of my site-wide hiera layers, I can override
>>>> ## the value without modifying the module or class declaration
>>>>
>>>> # /etc/puppet/hieradata/**snowflake.domain.com.yaml
>>>> webserver::vhost_dir: '/some/other/dir'
>>>>
>>>> This way the module author (who probably has the most work to do and
>>>> needs the expressiveness of the DSL) can provide default data, but site
>>>> administrators can still override it using mechanisms they're already
>>>> using.
>>>>
>>>> Note too that this is the next iteration, not necessarily the end
>>>> state. It's super important to get this right because the whole community
>>>> is going to have to live with it for a long time; those of you out here on
>>>> the bleeding edge willing to risk some skin to make something awesome are
>>>> critical to making that happen.
>>>>
>>>>
>>>> Eric Sorenson - [email protected] - freenode #puppet: eric0
>>>>
>>>> puppet platform // coffee // techno // bicycles
>>>>
>>>>
>>>> [puppetlabs-ntp]: https://github.com/apenney/**
>>>> puppetlabs-ntp/tree/data-in-**modules<https://github.com/apenney/puppetlabs-ntp/tree/data-in-modules>
>>>> [startrek]: https://github.com/pro-puppet/**puppet-module-startrek
>>>> [param-classes<https://github.com/pro-puppet/puppet-module-startrek%5Bparam-classes>]:
>>>>
>>>> http://docs.puppetlabs.com/**guides/parameterized_classes.**
>>>> html#appendix-smart-parameter-**defaults<http://docs.puppetlabs.com/guides/parameterized_classes.html#appendix-smart-parameter-defaults>
>>>> [15746]:
>>>> https://projects.puppetlabs.**com/issues/15746<https://projects.puppetlabs.com/issues/15746>
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Puppet Developers" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to puppet-dev+...@**googlegroups.com.
>>>> To post to this group, send email to [email protected].
>>>>
>>>> Visit this group at
>>>> http://groups.google.com/**group/puppet-dev<http://groups.google.com/group/puppet-dev>
>>>> .
>>>> For more options, visit
>>>> https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out>
>>>> .
>>>>
>>>
>>> --
>> You received this message because you are subscribed to the Google Groups
>> "Puppet Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected] <javascript:>.
>> To post to this group, send email to [email protected]<javascript:>
>> .
>> Visit this group at http://groups.google.com/group/puppet-dev.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>
>
> --
> Erik Dalén
>
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.
