On Thursday, May 15, 2014 9:55:05 PM UTC+2, John Bollinger wrote:
>
>
>
> On Thursday, May 15, 2014 9:46:40 AM UTC-5, Alessandro Franceschi wrote:
>>
>> Hallo everybody,
>> I've tried to look around in the group for past discussions about this
>> topic but haven't found any.
>> If this has been already debated , please forgive me and point me to the
>> right direction.
>>
>> I wonder what do you thing about a feature request to have data bindings
>> also for defines' parameters.
>>
>
>
> I'm skeptical about their value. We already have data bindings for
> classes, by which resource parameters can (indirectly) be injected, and I
> am inclined to think that classes are the right level of abstraction. On
> the other hand, we also have create_resources(), which already can give you
> resource-level data binding if you want to use it that way.
>
> I do not favor drawing distinctions between defined resource types and
> native resource types. On that basis, I would argue for data binding
> either for all resource types or for none, and against data binding for
> only defined types.
>
> I am concerned about the impact. It is already somewhat costly for Puppet
> to evaluate data bindings for class parameters, and adding bindings for
> resource parameters (even just for resources of defined types) will magnify
> that. Note that the cost scales with the aggregate number of defined
> parameters for all declared resources, independent whether any data are
> actually bound. In fact, the cases were no data are bound are the most
> costly, because hiera must then search the entire hierarchy.
>
Yes, these are valid and convincing points.
Anyway if we find data binding useful for classes and can bear the
performance overhead, I suppose we can do the same for defined types.
>
>
>
>> Naming is straightforward, a define like:
>> apache::vhost { 'example.com': }
>>
>> could have its params looked in this namespace:
>> apache::vhost::example.com::<parameter>
>>
>> My poor understanding of Puppet code points me to:
>>
>> https://github.com/puppetlabs/puppet/blob/master/lib/puppet/resource.rb#L343
>> and makes me guess that such a change should not be too difficult.
>>
>
>
> I agree, the change probably would not be difficult to implement -- just
> costly. It might be interesting to have some data on what proportion of
> invocations of that method follow the "return nil" branch, avoiding a
> lookup. As a wild guess, I'll say at least 75% in a typical catalog
> compilation run (maybe we can start a pool :-)). Even if it were only 25%,
> though, that would translate into a 33% increase in the number of
> (expensive) lookups.
>
>
>> I see many wonderful use cases for such a feature and no apparent cons.
>>
>
>
> I see no particularly good use cases and several cons, but I'm prepared to
> be amazed by your vision. Would you care to elaborate?
>
Yes, why not. Let me digress a bit then. It could be useful to verify if
the following idea seems cool only to me.
First, a premise.
I've written a lot of modules which are ageing and have large parts which
are very similar. I don't have the time to keep them all updated and there
are around better alternatives for many applications, even if sometimes
when I try to use modules from others I find myself screaming in despair.
For a long time I wondered how I could bypass all my code duplication, at
the same time I was more and more convinced that an application module
should be considered as a sort of library, to be used as is, without
modifications, (reusability), which have standard interfaces (naming
standards) and should not be opinionated in how in manages its application.
According to me opinionated modules can stay at higher abstraction levels
(profiles/stacks/whatever... they have to deliver a working setup, so some
choice on how things should be done is needed there) but the component
modules should just expose a clear and consistent interface on how to
manage the application they are made for.
Also I've seen that most of the times I found myself just with the need of
having a configuration file as wanted, eventually managing correlations and
integrations among (component) modules in custom higher abstraction classes
with dedicated templates and resources where all the logic could be defined
as I needed.
Finally I think that exposing as modules parameters each (or many) of the
managed application configuration entries is a dead end.
When I see a define like
this:
https://github.com/puppetlabs/puppetlabs-apache/blob/master/manifests/vhost.pp
with dozens of application specific parameters and the virtual host file
content managed with:
content => template('apache/vhost.conf.erb'),
instead of something like:
define apache::vhost(
$docroot,
$template = 'apache/vhost.conf.erb',
[...]
content => template($template),
I ask me a lot of weird questions. But I'm digressing. My point is that a
reusable module should allow total freedom on how its configurations should
be managed: with a file-based approach (source/template/ now
epp_template...), with a setting-based approach, (augeas, file_line..) with
concat or whatever. User should decide what's best for him, not the module
author.
Also I think that a single parameter that expects an hash ( options_hash,
config_file_options_hash, options, config... name it as you want) whose key
values can be freely used in a [custom] template is better than dozens of
parameters, totally unmaintainable, one for each possible configuration
entry of an application (and for each one, as you pointed, a data binding
lookup).
Given this premise (longer than expected, sorry), I've recently thought
about a single module: tp (stays for tiny puppet) which should be able to
manage essential features of any application.
Something you can use in manifests in a similar fashion:
To install an application (the tp module contains all the data to do it
right on different OS)
tp::install { 'redis': }
To configure it via a template:
tp::conf { 'redis.conf':
application => 'redis',
template => 'site/redis/redis.conf.erb',
}
Or, alternatively (using in the title any sane separator):
tp::conf { 'redis--redis.conf':
template => 'site/redis/redis.conf.erb',
}
To configure it via the fileserver:
tp::conf { 'redis.conf':
application => 'redis',
source => 'puppet:///modules/site/redis/redis.conf',
}
But also have something like (to manage single lines in a configuration
file):
tp::line { 'redis::redis.conf::port':
value => '1234',
}
or even something like:
tp::concat { 'redis':
target => 'redis.conf',
order => 10,
content => 'port 1234',
}
And manage, if overrides are needed, any internally used parameter with
something like:
tp::settings { 'redis':
lookup_strategy => 'merge', # Such a settings should define if the
module's data has to be merged or not with user data
settings => {
config_dir_path => '/opt/redis/conf',
tcp_port => '3242',
pid_file_path => '/opt/redis/run/redis.pid',
},
Also, it could be nice to have a face that allows commands like:
puppet tp check redis # To check is redis is running, based on settings
like service name , pid file, port ... (Incidentally this would make
integration tests a "bit" easier and quicker)
puppet tp info redis # To show info about how redis is managed by Puppet or
how it is working
Basically such a tp module would be a sum of most of my existing modules
main features + the puppi module (https://github.com/example42/puppi, for
the functionalities not related to application deployments).
Now, the real added value of such a thing would be a (Hiera like) set of
yaml files where all the default settings for as many applications as
possible for different operating systems can be defined in order to make
usage of the previous defines possible with a wide variety of applications
and allow, at the same time, extreme customization options.
To do this (hey, this is the topic of this thread), it would have been nice
to have a data in modules approach (as based on defines parameters) and
have in files like:
tp/data/redis/default.yaml , tp/data/redis/osfamily/RedHat.yaml all the
data needed for all the managed applications, such as :
---
tp::settings::redis::port: '6379'
tp::settings::redis::config_file_path: '/etc/redis/redis.conf'
Or, alternatively, data expressed in a format like :
---
redis::packages: (or tp::install::redis::packages...)
redis:
ensure: present
alias: 'redis'
redis::services:
redis:
ensure: running
enable: true
alias: 'redis'
require: Package[redis]
redis::files:
'/etc/redis/redis.conf':
ensure: present
owner: root
group: root
mode: 0644
alias: 'redis.conf'
notify: Service[redis]
'/etc/redis':
ensure: directory
owner: root
group: root
mode: 0755
alias: 'redis.dir'
notify: Service[redis]
In this case the tp::install code could be as simple as this POC:
define tp::install (
$packages = { } ,
$services = { } ,
$files = { } ,
$execs = { } ,
$users = { } ,
$configs = { } ,
) {
if $packages {
create_resources('package', $packages)
}
if $services {
create_resources('service', $services)
}
if $files {
create_resources('file', $files)
}
if $execs {
create_resources('exec', $execs)
}
if $users {
create_resources('user', $users)
}
}
Anyway, however is organized the internal data, and the relevant code
(instead of the create_resource we could use a lambda and cycle over the
various hashes) having the possibility to use Hiera for the backend would
be a good thing also because it would make easier for user to override the
default module data with custom one.
I guess there are better ways to obtain the same (suggestions welcomed)
rather than enabling data bindings for defined types and using data in
modules (both features being not existing in Puppet core). For example with
a custom function, but I have to figure out how to do it in the right way.
... but, what were we talking about? :-)
--
You received this message because you are subscribed to the Google Groups
"Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-dev/11178e96-9a30-4b89-9df9-5c31207ad9ce%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
