On Sun, Sep 09, 2007 at 11:22:26AM -0700, Bill Moseley wrote: > On Sun, Sep 09, 2007 at 12:28:58PM -0400, Peter Hartzler wrote: > > I'm hoping to reduce the overall memory footprint without losing the > > benefits of caching. (Lazy AND greedy!) Tmpl->Perl parsing overhead is > > less of an issue for us, for the reason you mention. > > > The templates have to be in memory -- they are just anonymous perl > subs, after all. Loading them into the parent process before forking > might help, but I'm not sure how to do that since you don't really > know what templates are going to be loaded until run-time.
Right, they have to be in memory *while in use*. Then they can be forgotten, e.g., not cached in TT's cache, and when needed again, pulled from the one central copy provided by memcached. The idea is to persist them, but only in one place, instead of across each and every apache child process. > I have also wondered about very text-heavy templates. It does seem > like a waste to have all that text in multiple process's memory space. > But, when I adding up all the text it wasn't really that much in > bytes. Good point. I definitely need to probe the issue more, and make sure I'm worrying about the right thing. It would be silly to do all this only to find out that the real memory consumption issue is something else entirely. > > It's entirely possible that other approaches might be better. I'm > > suspicious of my proposal if for no other reason than the leading '_' in > > '_load_compiled()' So, your questions, and the testing/verification of > > any answers, are quite valuable, thanks! > I'm not sure how significant the underscore is. The sense I get from the source is that underscore-prefixed routines are intended to be internal, and less intended to be overridden. Put another way, it's a tiny flag that suggests that there may be a better/cleaner more "true to TT" solution. > And placing the parsed template cache in memcached doesn't seem like > it would be helpful. I think using the local disk to store the > "compiled" templates would be better then fetching them over the > network. And again, fetching from that store should only happen when > first loading that template in each process. After that the template > should be cached in that LOOKUP hash. Just to be clear, my understanding is that each of possibly dozens of apache child process running on one machine will cache the compiled templates in their own space (leaving aside Randal's pre-fork slurp 'em all suggestion). It's this duplication I'm looking at. I was not entertaining the idea of serving memcached across ethernet -- If we did that, we'd probably just use the database server and be done with it. Except for the question of what, exactly, to cache, and where to hook it in. Pete. _______________________________________________ templates mailing list [email protected] http://lists.template-toolkit.org/mailman/listinfo/templates
