On 13 October 2013 12:40, Alessandro Franceschi <[email protected]> wrote:
> Thanks for the update Eric, very useful to understand the ongoing works on > data in modules. > > On Friday, October 11, 2013 9:01:19 PM UTC+2, Dan Bode wrote: > >> >> >> >> On Fri, Oct 11, 2013 at 11:09 AM, Eric Sorenson <[email protected] >> > wrote: >> >>> >>> Thanks to everyone who kicked the tires on the experimental data in >>> modules feature included in Puppet 3.3.0. We got a lot of feedback, some >>> cool proof-of-concept modules, and a definitive conclusion to the >>> experiment. >>> >>> The idea of including a module-specific hiera backend is centered around >>> one primary use case: replacing the 'params class pattern', a common idiom >>> in Puppet modules that's described in the [Using Parameterized >>> Classes][param-classes] guide. The problem that most testers ran into >>> though is that for non-trivial modules they ended up having to re-implement >>> the Puppet DSL logic encoded in their params.pp in convoluted, non-obvious >>> ways. The solutions to this led to more contortions until we'd ended up >>> with the ability to execute parser functions in the right-hand-side of a >>> yaml value. So something which started out trying to help separate data >>> from code ended up putting code back into data! >>> >>> Additionally, even after multiple attempts to simplify the surface area >>> and user experience with the bindings system (described in ARM-9) that >>> underlay the data-in-modules implementation, users still found its >>> complexity daunting. There are some important bits of scaffolding (like an >>> actual type system for Puppet!) that will prove valuable as more of the >>> future parser and evaluator work that Henrik is building makes its way into >>> the product, but in the final analysis the data in modules feature was the >>> wrong vehicle to introduce them. >>> >>> Refocusing on the problems users were trying to solve (and here I have >>> to give shout-outs to Ashley Penney for his [puppetlabs-ntp][] branch and >>> the dynamic duo of Spencer Krug/William van Hevelingen for their >>> [startrek][] module) and the problems with the 'params' pattern lent some >>> clarity. We've gotten into a situation of disparity with regard to hiera >>> and data bindings, because data bindings enable module _users_ to use their >>> site-wide hiera data but don't provide moduel _authors_ the same >>> affordance. But rather than introduce additional complexity, we can close >>> the gap for existing code patterns. >>> >>> So the proposed solution at this point is: >>> - enable an implicit data-binding lookup against the hiera-puppet >>> backend for a value of 'classname::variable' in the file >>> 'modules/classname/manifests/**params.pp', which simplifies class >>> definition and provides consistency with other hiera backends. As a module >>> author, you'd still leave your logic for variables in params.pp, but they'd >>> be implicitly looked up via data bindings as the class is declared, after >>> consulting site-wide hiera. >>> >> >> +1 >> >> Really happy to see this solved in a way that will not lead to complex >> migrations to Puppet 4. >> >> Although, to play devil's advocate, two concerns: >> >> the special nature of params as a namespace suffix: >> - how do users know not to use this namespace for anything else? >> - What if user declares resources in params? Does this fail? Do they >> always get realized when anything else from that namespace is applied? >> >> the magic mapping from <x>::parameter <x>::params::parameter may be >> something hard to grok for new users who are not already familiar with the >> params pattern. This is probably solvable with documentation and --debug >> logging, but still worth noting. >> > > Yes worth noting, but as you said proper documentation might suffice. > Anyway +1 also for me on the lookup to params.pp (Dan this, + Puppet 3 > data bindings, reminds me > https://github.com/example42/puppi/blob/master/lib/puppet/parser/functions/params_lookup.rb;-) > > I'd like to add to the complexity cases , in case you haven't still sorted > it out, the situations where some of the modules' params change according > to user provided values to other params. > For example look here: > > https://github.com/stdmod/puppet-elasticsearch/blob/master/manifests/init.pp#L124 > the path of the configuration dir ($dir param here) change according to > the value of the $install param, > and in order to manage this the code in the linked line is necessary. > I wonder how such a case could be managed with data in modules (without > replicating a similar logic). > Isn't that possible to solve by allowing one hiera value to use another hiera value for interpolation? https://projects.puppetlabs.com/issues/21367 > > al > > >> >>> - remove the user-facing '--binder' functionality >>> - fix known problems with the hiera-puppet lookups ([Redmine >>> 15746][15746], namely, but if there are others that are important to you >>> please speak up!) >>> >>> To show how this would work, I'll rework the ['smart parameter defaults' >>> example][param-classes] I linked above, with my commentary behind `##` >>> comments: >>> >>> # /etc/puppet/modules/webserver/**manifests/params.pp >>> >>> class webserver::params { ## nothing changes here... >>> $packages = $operatingsystem ? { >>> /(?i-mx:ubuntu|debian)/ => 'apache2', >>> /(?i-mx:centos|fedora|redhat)**/ => 'httpd', >>> } >>> $vhost_dir = $operatingsystem ? { >>> /(?i-mx:ubuntu|debian)/ => '/etc/apache2/sites-enabled', >>> /(?i-mx:centos|fedora|redhat)**/ => '/etc/httpd/conf.d', >>> } >>> } >>> >>> # /etc/puppet/modules/webserver/**manifests/init.pp >>> >>> class webserver( ## inheritance is gone, and >>> $packages, ## data bindings look up the defaults >>> $vhost_dir ## as webserver::params::vhost_dir >>> ) { >>> >>> package { $packages: ensure => present } >>> >>> file { 'vhost_dir': >>> path => $vhost_dir, >>> ensure => directory, >>> mode => '0750', >>> owner => 'www-data', >>> group => 'root', >>> } >>> } >>> >>> # /etc/puppet/manifests/site.pp >>> >>> node default { >>> class { 'webserver': } ## no params needed, they're in hiera >>> >>> ## then in one of my site-wide hiera layers, I can override >>> ## the value without modifying the module or class declaration >>> >>> # /etc/puppet/hieradata/**snowflake.domain.com.yaml >>> webserver::vhost_dir: '/some/other/dir' >>> >>> This way the module author (who probably has the most work to do and >>> needs the expressiveness of the DSL) can provide default data, but site >>> administrators can still override it using mechanisms they're already using. >>> >>> Note too that this is the next iteration, not necessarily the end state. >>> It's super important to get this right because the whole community is going >>> to have to live with it for a long time; those of you out here on the >>> bleeding edge willing to risk some skin to make something awesome are >>> critical to making that happen. >>> >>> >>> Eric Sorenson - [email protected] - freenode #puppet: eric0 >>> >>> puppet platform // coffee // techno // bicycles >>> >>> >>> [puppetlabs-ntp]: https://github.com/apenney/** >>> puppetlabs-ntp/tree/data-in-**modules<https://github.com/apenney/puppetlabs-ntp/tree/data-in-modules> >>> [startrek]: https://github.com/pro-puppet/**puppet-module-startrek >>> [param-classes<https://github.com/pro-puppet/puppet-module-startrek%5Bparam-classes>]: >>> http://docs.puppetlabs.com/**guides/parameterized_classes.** >>> html#appendix-smart-parameter-**defaults<http://docs.puppetlabs.com/guides/parameterized_classes.html#appendix-smart-parameter-defaults> >>> [15746]: >>> https://projects.puppetlabs.**com/issues/15746<https://projects.puppetlabs.com/issues/15746> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Puppet Developers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to puppet-dev+...@**googlegroups.com. >>> To post to this group, send email to [email protected]. >>> >>> Visit this group at >>> http://groups.google.com/**group/puppet-dev<http://groups.google.com/group/puppet-dev> >>> . >>> For more options, visit >>> https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out> >>> . >>> >> >> -- > You received this message because you are subscribed to the Google Groups > "Puppet Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/puppet-dev. > For more options, visit https://groups.google.com/groups/opt_out. > -- Erik Dalén -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/puppet-users. For more options, visit https://groups.google.com/groups/opt_out.
