I don't like any of the ideas I raised, but this will take some digging. We need to determine what life-time the cache should have, and what interface. I'm leaning towards either a cached read API in the FileSystem utilities, or a cache tied to the catalog lifetime.
On Tue, Dec 16, 2014 at 11:08 AM, Michael Smith < michael.sm...@puppetlabs.com> wrote: > > I'm trying to get my head around this, and I don't understand Henrik's > worry about unbound class variables causing memory leaks. If you have more > background on that, it might help. > > Avoiding globals in general means you have to have a file on disk. Any of > the solutions I can think of to speed that up - using an in-memory database > like SQLite or using a memory mapped file with > https://rubygems.org/gems/mmap (I have no experience with that module, so > not exactly a recommendation) - introduce their own global memory to keep > track of in-memory objects. However, they may be less problematic than > unbound class variables. > > Looking at how read_mounts is called, it looks pretty intrusive to try to > introduce a method argument to carry the cache. The other solution I think > is possible, but don't know off the top of my head how to do, is to change > the global variables to class variables that are bound when the module is > included; but that sounds like 'unbound class variables' to me. > > On Sun, Dec 7, 2014 at 3:12 PM, Trevor Vaughan <tvaug...@onyxpoint.com> > wrote: >> >> So, in PUP-3116, I proposed a (very bad) solution for reading >> /prod/mounts every time a file was opened during a run. >> >> The original goal was to get around cases where the sandbox service had >> not been properly configured and /proc/mounts grew to a few thousand lines. >> >> What I would *love* is a proper top-level queueing mechanism both for >> global items and for individual resources. >> >> Henrik indicated that the use of unbound class variables was causing >> memory leaks so that's out and he recommended the use of a JSON file on the >> system that would be used as a temporary cache. >> >> While this would work, I'm really not thrilled about having anything that >> we read thousands of time from disk. >> >> I did some tinkering and managed to bind an additional variable set to >> the catalog (being the only globally persistent thing that I could find) >> but that's obviously also a Very Bad(tm) solution. >> >> So, a couple of questions: >> >> 1) Do any of the client internals gurus have a suggestion as to how to do >> this properly. >> >> 2) If there's no way to do it properly, whats the most system-efficient >> way to do it successfully, albeit improperly. >> >> I'm not going to post the catalog shim code unless pressed because I fear >> being stoned at the next PuppetConf :-D. >> >> Thanks, >> >> Trevor >> >> -- >> Trevor Vaughan >> Vice President, Onyx Point, Inc >> (410) 541-6699 >> tvaug...@onyxpoint.com >> >> -- This account not approved for unencrypted proprietary information -- >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to puppet-dev+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoWGzUtfJW-oz62ByjPGFMye%2BCU8XbT2j3YRtoqgHRY-1A%40mail.gmail.com >> <https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoWGzUtfJW-oz62ByjPGFMye%2BCU8XbT2j3YRtoqgHRY-1A%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CABy1mMLzyurYms0LLVqW0BiEup%3DrnMmv6FafWQom0-WQDL5%2BNg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
