On Thursday, January 24, 2013 5:24:58 AM UTC-6, Daniel wrote:
>
> Seems that chaining exported resources might not be too efficient and
> produces lots of data that could be the reason for puppetdb crashing.
>
> The culprits being these two lines in two manifest files:
>
> ./nsca/server.pp: #File <<| tag == $get_tag |>> -> Nagios_host <<| tag ==
> $get_tag |>>
> ./nrpe/server.pp: #File <<| tag == $get_tag |>> -> Nagios_host <<| tag ==
> $get_tag |>>
>
>
And there may be your combinatorial problem. Supposing that the tags are
not specific to the exporting node, the numbers of Files and Nagios_hosts
increase linearly with the number of nodes, but the number of relationships
among them increases with the square of the number of nodes.
> replacing them with unchained:
>
> File <<| tag == $get_tag |>>
> Nagios_host <<| tag == $get_tag |>>
>
> causes it to run even with 1GB for puppetdb (still a 16GB vm) in under 10
> mins, which is acceptable.
>
Do you in fact need the order of application constraints that the chaining
declared? All of them?
This is an area that seems to give people problems. On one hand, Puppet
provides a couple of mechanisms that allow people to compactly declare
large numbers of ordering relationships. On the other hand, people
sometimes lose sight of the distinction between functional dependency and
application-order dependency. Either one of these can cause trouble, as
unneeded declarations of any kind slow Puppet and make it consume more
resources. They also make you manifests more susceptible to dependency
cycles. The combination is worse, however, because the number of unneeded
relationships produced can be multiplicative.
Depending on what relationships you actually need, you have various options:
1) If it doesn't actually matter whether Puppet syncs given Files before or
after it syncs any particular Nagios_host, then you never needed any
relationships at all, and simply removing the chaining as you did is the
best solution.
2) If specific Files must be synced before specific Nagios_hosts, then you
should express those relationships and not any others. In particular, if
you only need relationships among Files and Nagios_hosts exported by the
same node, then you should declare the relationships on the exporting side
instead of on the collecting side. (And if they always come in such pairs
then perhaps you should wrap then in a defined type and export that
instead).
3) If you really need something along the lines that the chaining expressed
-- all Files with a given tag synced before all Nagios_hosts bearing that
tag -- then you can break the combinatorial problem by introducing an
artificial barrier resource for each tag:
define mymodule::barrier() {
# no content
}
mymodule::barrier { $get_tag: }
File<<| tag == $get_tag |>> -> Mymodule::Barrier[ $get_tag ] -> Nagios_host
<<| tag == $get_tag |>>
That gives you (number(Files) + number(Nagios_host)) relationships for each
tag instead of (number(Files) * number(Nagios_host)) relationships.
Of course, you can use any single resource as a barrier; it doesn't need to
be of a defined type. In some cases there might be a real resource that
naturally fills the role. If you have Puppet's stdlib module then the
Anchor resource would be a good fit; it is intended for a similar purpose
anyway.
John
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/2_AvVt-Qm4oJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
