Issue #4499 has been updated by R.I. Pienaar aka Volcane.
James Turnbull wrote: > Previously it wasn't in core - we put extlookup in core in 2.6.0 hence we now > have to deal with it before it was RI's problem. I did contemplate later on how to make it cache some of the data but it's not really clear how functions should store persistent cache data. Caching data i think is a common case though esp when querying remote sources like api's so would be nice to see this specifically catered for in puppet through some utility classes ---------------------------------------- Refactor #4499: extlookup parses files multiple times http://projects.puppetlabs.com/issues/4499 Author: Alan Barrett Status: Needs more information Priority: Normal Assigned to: James Turnbull Category: functions Target version: queued Affected version: 0.25.5 Branch: According to "ruby -r profile", more than 50% of the runtime for "puppetmasterd --compile someclient" is in CSV#parse_body. In a typical example, extlookup is called 204 times, CSV::Reader#each is called 225 times (only a little more than the number of calls to extlookup, because almost all variables are found in the first file), CSV#parse_row is called 42486 times (which is not much less than the product of {number of lines in the file} * {number of calls to extlookup}), and CSV#parse_body is called 84747 times (which is approximately double the number of calls to CSV#parse_body). I The code in extlookup.rb appears to read and parse the data files on every call. It would be better if it kept a cache of {file name, variable name, raw value} tuples, invalidated the cache when the file timestamp changed, and otherwise avoided unnecessary re-reading of unchanged files. The decision about which files to search should probably still be done on every call, in case extlookup_datadir or extlookup_precedence has changed. Expansion of "#{variable}" embedded in the raw value should also be done on every call. -- You have received this notification because you have either subscribed to it, or are involved in it. To change your notification preferences, please click here: http://projects.puppetlabs.com/my/account -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-bugs?hl=en.
