Thanks for the update Eric, very useful to understand the ongoing works on data in modules.
On Friday, October 11, 2013 9:01:19 PM UTC+2, Dan Bode wrote: > > > > > On Fri, Oct 11, 2013 at 11:09 AM, Eric Sorenson > <[email protected]<javascript:> > > wrote: > >> >> Thanks to everyone who kicked the tires on the experimental data in >> modules feature included in Puppet 3.3.0. We got a lot of feedback, some >> cool proof-of-concept modules, and a definitive conclusion to the >> experiment. >> >> The idea of including a module-specific hiera backend is centered around >> one primary use case: replacing the 'params class pattern', a common idiom >> in Puppet modules that's described in the [Using Parameterized >> Classes][param-classes] guide. The problem that most testers ran into >> though is that for non-trivial modules they ended up having to re-implement >> the Puppet DSL logic encoded in their params.pp in convoluted, non-obvious >> ways. The solutions to this led to more contortions until we'd ended up >> with the ability to execute parser functions in the right-hand-side of a >> yaml value. So something which started out trying to help separate data >> from code ended up putting code back into data! >> >> Additionally, even after multiple attempts to simplify the surface area >> and user experience with the bindings system (described in ARM-9) that >> underlay the data-in-modules implementation, users still found its >> complexity daunting. There are some important bits of scaffolding (like an >> actual type system for Puppet!) that will prove valuable as more of the >> future parser and evaluator work that Henrik is building makes its way into >> the product, but in the final analysis the data in modules feature was the >> wrong vehicle to introduce them. >> >> Refocusing on the problems users were trying to solve (and here I have to >> give shout-outs to Ashley Penney for his [puppetlabs-ntp][] branch and the >> dynamic duo of Spencer Krug/William van Hevelingen for their [startrek][] >> module) and the problems with the 'params' pattern lent some clarity. We've >> gotten into a situation of disparity with regard to hiera and data >> bindings, because data bindings enable module _users_ to use their >> site-wide hiera data but don't provide moduel _authors_ the same >> affordance. But rather than introduce additional complexity, we can close >> the gap for existing code patterns. >> >> So the proposed solution at this point is: >> - enable an implicit data-binding lookup against the hiera-puppet backend >> for a value of 'classname::variable' in the file >> 'modules/classname/manifests/params.pp', which simplifies class definition >> and provides consistency with other hiera backends. As a module author, >> you'd still leave your logic for variables in params.pp, but they'd be >> implicitly looked up via data bindings as the class is declared, after >> consulting site-wide hiera. >> > > +1 > > Really happy to see this solved in a way that will not lead to complex > migrations to Puppet 4. > > Although, to play devil's advocate, two concerns: > > the special nature of params as a namespace suffix: > - how do users know not to use this namespace for anything else? > - What if user declares resources in params? Does this fail? Do they > always get realized when anything else from that namespace is applied? > > the magic mapping from <x>::parameter <x>::params::parameter may be > something hard to grok for new users who are not already familiar with the > params pattern. This is probably solvable with documentation and --debug > logging, but still worth noting. > Yes worth noting, but as you said proper documentation might suffice. Anyway +1 also for me on the lookup to params.pp (Dan this, + Puppet 3 data bindings, reminds me https://github.com/example42/puppi/blob/master/lib/puppet/parser/functions/params_lookup.rb ;-) I'd like to add to the complexity cases , in case you haven't still sorted it out, the situations where some of the modules' params change according to user provided values to other params. For example look here: https://github.com/stdmod/puppet-elasticsearch/blob/master/manifests/init.pp#L124 the path of the configuration dir ($dir param here) change according to the value of the $install param, and in order to manage this the code in the linked line is necessary. I wonder how such a case could be managed with data in modules (without replicating a similar logic). al > >> - remove the user-facing '--binder' functionality >> - fix known problems with the hiera-puppet lookups ([Redmine >> 15746][15746], namely, but if there are others that are important to you >> please speak up!) >> >> To show how this would work, I'll rework the ['smart parameter defaults' >> example][param-classes] I linked above, with my commentary behind `##` >> comments: >> >> # /etc/puppet/modules/webserver/manifests/params.pp >> >> class webserver::params { ## nothing changes here... >> $packages = $operatingsystem ? { >> /(?i-mx:ubuntu|debian)/ => 'apache2', >> /(?i-mx:centos|fedora|redhat)/ => 'httpd', >> } >> $vhost_dir = $operatingsystem ? { >> /(?i-mx:ubuntu|debian)/ => '/etc/apache2/sites-enabled', >> /(?i-mx:centos|fedora|redhat)/ => '/etc/httpd/conf.d', >> } >> } >> >> # /etc/puppet/modules/webserver/manifests/init.pp >> >> class webserver( ## inheritance is gone, and >> $packages, ## data bindings look up the defaults >> $vhost_dir ## as webserver::params::vhost_dir >> ) { >> >> package { $packages: ensure => present } >> >> file { 'vhost_dir': >> path => $vhost_dir, >> ensure => directory, >> mode => '0750', >> owner => 'www-data', >> group => 'root', >> } >> } >> >> # /etc/puppet/manifests/site.pp >> >> node default { >> class { 'webserver': } ## no params needed, they're in hiera >> >> ## then in one of my site-wide hiera layers, I can override >> ## the value without modifying the module or class declaration >> >> # /etc/puppet/hieradata/snowflake.domain.com.yaml >> webserver::vhost_dir: '/some/other/dir' >> >> This way the module author (who probably has the most work to do and >> needs the expressiveness of the DSL) can provide default data, but site >> administrators can still override it using mechanisms they're already using. >> >> Note too that this is the next iteration, not necessarily the end state. >> It's super important to get this right because the whole community is going >> to have to live with it for a long time; those of you out here on the >> bleeding edge willing to risk some skin to make something awesome are >> critical to making that happen. >> >> >> Eric Sorenson - [email protected] <javascript:> - freenode >> #puppet: eric0 >> puppet platform // coffee // techno // bicycles >> >> >> [puppetlabs-ntp]: >> https://github.com/apenney/puppetlabs-ntp/tree/data-in-modules >> [startrek]: https://github.com/pro-puppet/puppet-module-startrek >> [param-classes<https://github.com/pro-puppet/puppet-module-startrek%5Bparam-classes>]: >> >> >> http://docs.puppetlabs.com/guides/parameterized_classes.html#appendix-smart-parameter-defaults >> [15746]: https://projects.puppetlabs.com/issues/15746 >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To post to this group, send email to [email protected]<javascript:> >> . >> Visit this group at http://groups.google.com/group/puppet-dev. >> For more options, visit https://groups.google.com/groups/opt_out. >> > > -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/puppet-users. For more options, visit https://groups.google.com/groups/opt_out.
