Issue #17230 has been reported by Josh Shabtai.
----------------------------------------
Bug #17230: Chaining resource collections is unnecessarily slow
https://projects.puppetlabs.com/issues/17230
Author: Josh Shabtai
Status: Unreviewed
Priority: Normal
Assignee:
Category:
Target version:
Affected Puppet version:
Keywords:
Branch:
Hey,
Puppet master version: 2.7.18
Puppet agent version: 2.7.1
It seems that when chaining resource collections (e.g. <|collection1|> ->
<|collection2|>), puppet creates dependencies between each resource in each
side. If, for example, you have 100 of each resource, this creates 10,000
dependencies and seems to take a long time to handle. We have a situation
similar to this (though I don't know exactly how many resources we actually
have). I added a dummy resource in between each pair of collections though,
which sped things up significantly (e.g. <|collection1|> -> Exec["dummy"] ->
<|collection2|>). I would suggest that you do something similar internally.
Here's the diff that got my runtime from over an hour to about half that:
`+ # Dummy resources for chaining
+ # The idea is that when puppet chains collections of sizes x and y, you get
+ # x*y requirement. By adding a dummy resource in the middle, we should be
+ # able to get it down to x+y, which will drastically speed things up.
+ exec { "dummy-domain-subdomain": command => "/bin/true" }
+ exec { "dummy-mailbox-set-status": command => "/bin/true" }
+ exec { "dummy-set-status-child-mailbox": command => "/bin/true" }
+ exec { "dummy-child-mailbox-set-cos": command => "/bin/true" }
# Resource chaining
- Exec["cache domains"] -> Exec<| tag == "domain" |> -> Exec<| tag ==
"subdomain" |> -> Exec["cache cos"] -> Exec<| tag == "cos" |> -> Exec["cache
mailboxes"] -> Exec<| tag == "mailbox" |> -> Exec<| tag == "set-status" |> ->
Exec<| tag == "child-mailbox" |> -> Exec<| tag == "set-cos" |>
+ Exec["cache domains"] -> Exec<| tag == "domain" |> ->
Exec["dummy-domain-subdomain"] -> Exec<| tag == "subdomain" |> -> Exec["cache
cos"] -> Exec<| tag == "cos" |> -> Exec["cache mailboxes"] -> Exec<| tag ==
"mailbox" |> -> Exec["dummy-mailbox-set-status"] -> Exec<| tag == "set-status"
|> -> Exec["dummy-set-status-child-mailbox"] -> Exec<| tag == "child-mailbox"
|> -> Exec["dummy-child-mailbox-set-cos"] -> Exec<| tag == "set-cos" |>`
--
You have received this notification because you have either subscribed to it,
or are involved in it.
To change your notification preferences, please click here:
http://projects.puppetlabs.com/my/account
--
You received this message because you are subscribed to the Google Groups
"Puppet Bugs" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/puppet-bugs?hl=en.
