On Wednesday, December 18, 2013 4:01:24 AM UTC+1, Eric Sorenson wrote: > > Alessandro Franceschi wrote: > > Hi all, > > I suppose most of you have read R.I.Pienaar's blog post about his > > module_data module: > > > http://www.devco.net/archives/2013/12/08/better-puppet-modules-using-hiera-data.php?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+planetpuppet+%28Planet+Puppet%29 > > > > > I've played a bit with it still haven't found any issue. > > It's very easy and straightforward to implement (let's be honest, ARM-9 > > approach was not as plainly easy) > > This commit shows how quickly a module can be converted: > > http://bit.ly/18K6dSY > > This is great Alessandro, I actually hadn't seen that module-skeleton > before. > > > But the real advantage, at least for me, is not for such a common case > > (where params.pp may not be elegant but does what's needed), but a case > > where some values which typically go in params.pp may change not > > according to the os but according to parameters passed to the module, > > such as installation method or version. > > > > In such a case I found this approach really a killer. > > An example is here: http://bit.ly/1gCkQQ8 > > I see that it moved the params into multiple yaml files. I guess this is > the core of it: if you see data in yaml as being superior, then having > more ubiquitous hiera+yaml seems like a natural fit. If you like the DSL > and programming-language features such as string interpolation, you > either end up putting those features into YAML (like the Hiera 1.3 > lookup function) or massaging the return values once you're back in the > DSL.
> > So, my point is: > > if it's true (that's how I've understood) that the data in module > > solution introduced in 3.3.0 has been held back, is there any chance > > that this one might be introduced into core Puppet? > > > > And, in any case, what's the current plan for Data in modules? > > So let me recap from the start and talk a bit about where I think it's > going. Honestly, this is kind of a painful mail to write because in a > way it is a litany of my failure to manage this feature effectively. > > RI's original pull request had a couple of minor issues that are in the > #20-#30 update range on #16856 and could have easily been fixed up after > merge. I should not have let it be overshadowed by the Binder > implementation and should not have let that be merged in without > following the ARM process, which we'd introduced in between the first > talk about this feature and the later developments. Looking back, this > merge caused a big problem because it created an unfair playing field > between competing implementations. > > Then, the 3.3.0 implementation (Binder) got hammered in user testing and > then I failed at prioritizing its removal in time for 3.4.0. So that > implementation is in kind of Purgatory until the tickets to clean it up > can be addressed (these are linked under > https://tickets.puppetlabs.com/browse/PUP-42 ). The discussion from > October led to the tentative plan to introduce a simple, hiera-based DSL > data binding lookup but clearly I did not incorporate the demand for > YAML based data storage and, in any case, the development time for it > also got squeezed out as we got close to 3.4.0. So here we are. > > A semi-ordered list of next steps: > - me: migrate the original ticket to JiRA to preserve history and invite > further comment - done, now at > https://tickets.puppetlabs.com/browse/PUP-1157<https://www.google.com/url?q=https%3A%2F%2Ftickets.puppetlabs.com%2Fbrowse%2FPUP-1157&sa=D&sntz=1&usg=AFQjCNEHKdbeVc_mVLzWZSoaqcKvBRNkAg> > > > - community: please try RI's module_data in your modules and write up > your experience, especially with complex modules as I'm very worried > about the problem that Gary Larizza describes on his blog post about > deep hiera hierarchies becoming hard to reason about ( > http://garylarizza.com/blog/2013/12/08/when-to-hiera/<http://www.google.com/url?q=http%3A%2F%2Fgarylarizza.com%2Fblog%2F2013%2F12%2F08%2Fwhen-to-hiera%2F&sa=D&sntz=1&usg=AFQjCNEwxC9dT-XzRdS_wec3PxqBNEqHnA>under > "What are > you trying to say"). I've talked to several sysadmins at large sites who > went all-in with Hiera and ended up with >15-layer hierarchies that were > incomprehensibly complex to the rest of their team (in a way, hierarchy > layout is just a super-compressed 'if' statement and the same problems > apply). > I'll for surely test Rip's solution more, but for how I can see te whole matter the hiera hierarchies proliferation is most of the times due to a wrong approach towards modules: as Gary underlined, all OS specific stuff should be in module domain (that's how I conceive a reusable module and the first requirement for reusability is multiple OS support) and therefore in an eventual in module data hierarchy. All the business logic , which varies on each site and can't be standardised, is managed in the main hiera hierarchy. To be honest most of the (also official) documentation examples about sample hierarchies in hiera.yaml, where most of the times where used os related facts, has not helped people in understanding this ( remember a, mostly-ignored-as-usual, comment I did on this old post about hiera: http://puppetlabs.com/blog/first-look-installing-and-using-hiera ). About the internal module hierarchy up to now I haven't found situations where I needed to manage something more that osfamily/operatingsystem/operatingsystemversion differences, the only notable difference is in cases where different installation options (and relevant class parameters) might involve an extra layer in the module's hierarchy (like the %{install_class} in the sample http://bit.ly/1gCkQQ8 ) If someone has more complex cases please let's review them and see if they can be managed with RIP's module, considering that, as he himself said in other posts, we don't have necessarily have to cover the 100% of use cases, for corner situations nobody prevents us to keep on using the solutions available from Puppet DSL. > > - all: try to work through the problems we know we have in data bindings > and hiera on the way to a complete solution, i.e. > https://tickets.puppetlabs.com/browse/HI-118<https://www.google.com/url?q=https%3A%2F%2Ftickets.puppetlabs.com%2Fbrowse%2FHI-118&sa=D&sntz=1&usg=AFQjCNETa4dnL-x_XUMucXjuQgVUW1Pqog> > > https://tickets.puppetlabs.com/browse/HI-46<https://www.google.com/url?q=https%3A%2F%2Ftickets.puppetlabs.com%2Fbrowse%2FHI-46&sa=D&sntz=1&usg=AFQjCNEfGHuGFju7OKOdkq8tkQ81ZuvYXw> > > and the general problem of expressing a dependency on the module_data > backend in a consuming module's metadata (today, being in a module is an > improvement over it being in core, because a DIM-enabled consumer can > explicitly state that dependency, which obviously isn't the greatest). > > - me: stay on top of communicating these developments to a far better > degree so we're not still in this situation 6 months down the line. I'd > also like to make sure we're all trying solve the same problem and will > start up a google doc to facilitate commenting and collaboration on > goals and acceptance criteria. > > Hope this helps. > Definitively. Thank you for the exhaustive reply, hope a solution to this core feature for modules will be sorted out in reasonable times. Hope also that whatever situation may arise it will be possible to use it also on earlier Puppet versions (as is Rip's module) . Al -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/fab78cd7-7804-42a2-8b0c-488b99ba2afd%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
