Hi Luke. Thanks for a thoughtful and detailed response.
Quite a similar question was posted about two weeks back, you might find
> that very interesting:
> https://groups.google.com/forum/#!topic/puppet-users/NW2yuHMJvsY
>
I saw this, and have been kicking around the idea leading to this post
If you are a confident Puppet Coder, you might prefer to import the source,
> patch the module to add your feature, then submit the patch back upstream.
>
This is likely part of my problem. I am not a confident puppet coder,
probably closer to barely competent.
> When using roles and profiles you end up declaring all the module
>> parameters again to avoid losing functionality and flexibility.
>>
>
> ... Not sure I agree with that statement. That sounds odd. Why would you
> be re-declaring module parameters if you're not changing something from the
> defaults? And if you are intending to change something, then of course you
> are supplying different parameters?
>
Lets say a module has 10 parameters and supplies defaults for most of
them. When writing a profile you have to choose how many of the class
parameters can remain defaults, how many to override, and how many to
expose as profile parameters. It's sounds fine to limit the number of
parameters at the profile, right up until you hit an edge case that doesn't
work with the default values and the parameter you need to change now
requires a profile update...
> You also need to be familiar with all the classes, types, and parameters
>> from all modules in order to use them effectively.
>>
>
> Ideally the README page of a module would contain amazing user level
> documentation of how the module should work... but not that many do. I
> often find I have to go read the Puppet code itself to figure out exactly
> what a parameter does.
>
Ditto on the documentation. Some modules are better than others, and of
course you can review the manifests, but with my admitted weakness in
Puppet DSL it's not always immediately apparent to me what some classes are
doing.
> To avoid all of the above, I put together the 'basic' module and posted it
>> on the forge: https://forge.puppet.com/southalc/basic
>>
>
> Ok :-) I'm beginning to see what the core of your problem is. The fact
> that you've created your own module to effectively do create_resources()
> hash definitions says to me that you haven't quite grasped the concepts of
> the Role / Profile design pattern. I know I have a very strong view on
> this subject and many others will disagree, but personally I think the Role
> / Profile pattern and the "do-everything-with-Hiera-data" pattern are
> practically incompatible.
>
I'd like to think I grasp the roles/profiles concept, but am just not
convinced it's a better approach. Abstracting away configuration details
and exposing a limited set of parameters results in uniform
configurations. In doing so it also seems it limits flexibility and
ensures that you'll continue to spend a good deal of time maintaining your
collection of profiles/modules.
> This module uses the hiera_hash/create_resources model for all the native
>> puppet (version 5.5) types, using module parameters that match the type
>> (exceptions for metaparameters, per the README). The module also includes
>> the 'file_line' type from puppetlabs/stdlib, the 'archive' type from
>> puppet/archive, and the local defined type 'binary', which together provide
>> a simple and powerful way to create complex configurations from hiera. All
>> module parameters default to an empty hash and also have a merge strategy
>> of 'hash' to enable a great deal of flexibility. With this approach I've
>> found it possible to replace many single purpose modules it's much faster
>> and easier to get the results I'm looking for.
>>
>
> A Hiera-based, data-driven approach will always be faster to produce a
> "new" result (just like writing Ansible YAML is faster to produce than
> Puppet code)... It's very easy to brain dump configuration into YAML and
> have it work, and that's efficient up to a certain point. For your simple
> use cases, yes, I can completely see why you would be looking at the Role
> Profile pattern and saying to yourself "WTF for?". I think the tipping
> point of which design method becomes more efficient directly relates to how
> complicated (or how much control) you want over your systems.
>
A number of people I've talked to like Ansible because of the easy learning
curve and great time-to-results.
> The more complicated you go, the more I think you will find that Hiera
> just doesn't quite cut it. Hiera is a key value store. You can start
> using some neat tricks like hash merging, you can look up other keys to
> de-duplicate data... When you start to model more and more complicated
> infrastructure, I think you will find that you don't have enough power in
> Hiera to describe what you want to describe, and that you need an
