Issue #4499 has been updated by Nigel Kersten. Assignee deleted (Markus Roberts)
I still don't have any picture on why caching would hurt in this case. ---------------------------------------- Refactor #4499: extlookup parses files multiple times https://projects.puppetlabs.com/issues/4499 Author: Alan Barrett Status: Needs More Information Priority: Normal Assignee: Category: functions Target version: Affected Puppet version: 0.25.5 Keywords: Branch: According to "ruby -r profile", more than 50% of the runtime for "puppetmasterd --compile someclient" is in CSV#parse_body. In a typical example, extlookup is called 204 times, CSV::Reader#each is called 225 times (only a little more than the number of calls to extlookup, because almost all variables are found in the first file), CSV#parse_row is called 42486 times (which is not much less than the product of {number of lines in the file} * {number of calls to extlookup}), and CSV#parse_body is called 84747 times (which is approximately double the number of calls to CSV#parse_body). I The code in extlookup.rb appears to read and parse the data files on every call. It would be better if it kept a cache of {file name, variable name, raw value} tuples, invalidated the cache when the file timestamp changed, and otherwise avoided unnecessary re-reading of unchanged files. The decision about which files to search should probably still be done on every call, in case extlookup_datadir or extlookup_precedence has changed. Expansion of "#{variable}" embedded in the raw value should also be done on every call. -- You have received this notification because you have either subscribed to it, or are involved in it. To change your notification preferences, please click here: http://projects.puppetlabs.com/my/account -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-bugs?hl=en.
