I'm doing my own digging to figure out what seems to make sense. Josh had mentioned Puppet::push_context, set in the configurer. We push and pop context for each apply run; however that's a private API that doesn't seem to be meant for general use. Piggybacking on it looks like it would get messy.
There's also Puppet::Util::Storage, which superficially looks appropriate for this kind of caching ( http://www.rubydoc.info/gems/puppet/Puppet/Util/Storage). I'm still trying to wrap my head around what side-effects might occur. On Tue, Dec 16, 2014 at 6:27 PM, Trevor Vaughan <tvaug...@onyxpoint.com> wrote: > > Part of my other heartburn with using a file was revisited hard upon me as > I recalled the original extdata function implementation. > > In the case of extdata, one large extdata file + a lot of extlookups = > massive catalog compile times on the server. > > So, every time I want to call the cache, across potentially large numbers > of providers and/or other things requiring state, I *really* don't want to > read a file. Particularly, when I don't know what's going to be in it. > > In this case, we would have to contend with slower client run times and > more CPU overhead as well as disk I/O requirements. Indicating that people > should change the way their OS is configured inasmuch as using tmpfs when > they may not have this choice does not seem ideal unless, of course, it > ships with puppet and doesn't require a system reboot. If, for some reason, > I have 50 providers that want to use this, this is 50 file reads and writes > that could be avoided. > > Giving people the choice of Disk vice Memory overhead would be ideal if > you want both for some reason. > > I'm honestly not seeing what would be so bad about scope.cache where cache > is some top level Puppet::Cache object that holds hashes that expire at the > end of a run. You would have to do things very politely in terms of > namespacing but you have to do that anyway. > > I am, of course, not opposed to saving cache state to disk for debugging > purposes, and think that should be an option when the --debug flag is used. > > Trevor > > Trevor > > On Tue, Dec 16, 2014 at 7:37 PM, Felix Frank < > felix.fr...@alumni.tu-berlin.de> wrote: >> >> Hey, >> >> good points - state retention at whatever granular level would be a good >> general purpose tool to have. If it's built in a pervasive fashion (i.e., >> any provider might use the cache for whetever it deems appropriate), it >> gains added visibility and becomes more opaque to the user - which is a >> good thing, and addresses one of the major concerns I'm having with this. >> The other being that it needs to be tunable for the user in some fashion. >> >> I have no qualms about disk I/O - after all, the user can choose whatever >> block backend they want. Users who depend on low latency or need to save >> IOPS can employ a tmpfs, for example. >> >> Cheers, >> Felix >> >> On 12/17/2014 12:56 AM, Trevor Vaughan wrote: >> >> I'm happy with catalog lifetime. >> >> I'm really not happy with doing anything that involves disk I/O. >> >> This would be key to getting providers to be able to save state in a >> non-hacky way as well. >> >> Trevor >> >> On Tue, Dec 16, 2014 at 6:45 PM, Michael Smith < >> michael.sm...@puppetlabs.com> wrote: >>> >>> I don't like any of the ideas I raised, but this will take some digging. >>> We need to determine what life-time the cache should have, and what >>> interface. I'm leaning towards either a cached read API in the FileSystem >>> utilities, or a cache tied to the catalog lifetime. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to puppet-dev+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/puppet-dev/5490D048.7020702%40Alumni.TU-Berlin.de >> <https://groups.google.com/d/msgid/puppet-dev/5490D048.7020702%40Alumni.TU-Berlin.de?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > > -- > Trevor Vaughan > Vice President, Onyx Point, Inc > (410) 541-6699 > tvaug...@onyxpoint.com > > -- This account not approved for unencrypted proprietary information -- > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to puppet-dev+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoUCo4FmT9QGk_P1kYg0CdEWA9pqhU%3D6jeXjBAr9z7fD9w%40mail.gmail.com > <https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoUCo4FmT9QGk_P1kYg0CdEWA9pqhU%3D6jeXjBAr9z7fD9w%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CABy1mM%2B2rP9wE%3D6bea91B5J_e6PLfNqjMB4KLWTLS6gT5iuxXg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
