Re: [Zope-dev] RAMcache and container vs. context
Stefan H. Holek wrote at 2005-5-27 10:59 +0100: >A TALES expression may be prohibitively expensive in any case, no >matter how simple it is kept. Please make sure to do some comparative >profiling. Cache keys are recomputed on every call of the script, >AFAICS. The thought of doing this in restricted Python makes my skin >crawl. But "TALES" expressions need not necessarily be restricted (see my "TrustedExecutables"). Though, I did not think about the security inplications when we would use unrestricted TALES expressions for a cache manager. It would probably be a bad idea. -- Dieter ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RAMcache and container vs. context
A TALES expression may be prohibitively expensive in any case, no matter how simple it is kept. Please make sure to do some comparative profiling. Cache keys are recomputed on every call of the script, AFAICS. The thought of doing this in restricted Python makes my skin crawl. Stefan On 26. Mai 2005, at 09:12, Chris Withers wrote: Paul Winkler wrote: I kinda wish the RAMCache manager allowed other things than REQUEST variables for differentiation. Arbitrary TALES expressions would be nice. Then we could get rid of this hard-coded gunk and get the same effect by having context/getPhysicalPath as one of the expressions. But of course then you'd have the opportunity to kill your cache performance by making the expressions too expensive. Ah well, TANSTAAFL and all that. I'll probably just drop the idea. I still think it's worth it. Crippling everyone on the basis that stupid people can hurt themselves is not good logic in my books ;-) If you could chuck this as a feature request in the collector and assign it to me, I'd love to work on it if I ever find the time... cheers, Chris -- Software Engineering is Programming when you can't. --E. W. Dijkstra -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RAMcache and container vs. context
On Thu, May 26, 2005 at 09:12:30AM +0100, Chris Withers wrote: > If you could chuck this as a feature request in the collector and assign > it to me, I'd love to work on it if I ever find the time... Done. http://www.zope.org/Collectors/Zope/1795 -- Paul Winkler http://www.slinkp.com ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RAMcache and container vs. context
Paul Winkler wrote: I kinda wish the RAMCache manager allowed other things than REQUEST variables for differentiation. Arbitrary TALES expressions would be nice. Then we could get rid of this hard-coded gunk and get the same effect by having context/getPhysicalPath as one of the expressions. But of course then you'd have the opportunity to kill your cache performance by making the expressions too expensive. Ah well, TANSTAAFL and all that. I'll probably just drop the idea. I still think it's worth it. Crippling everyone on the basis that stupid people can hurt themselves is not good logic in my books ;-) If you could chuck this as a feature request in the collector and assign it to me, I'd love to work on it if I ever find the time... cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RAMcache and container vs. context
On Mon, Apr 25, 2005 at 03:19:42PM -0300, Leonardo Rochael Almeida wrote: > Sorry for comming late into the fray :-) (snip) > This alternative won't help you when: > > * the pythonscript is sensitive to the context AND > > * it is called w/ 2 or more different contexts on the same request > > An artificial example I can come up w/ from the top of my head: > Suppose you have a folder w/ an index_html that displays information > about it's subfolders that is calculated by an expensive script and you > want to cache the results of this script, you'd go: > > > > Folder contains > > shruberries. > > > > Then you'd go and RAMCache "expensively_calculate_shruberries" > > But if we implement the change you're suggesting, then this page would > list the same number of shruberries of the first subfolder for all of > them. > > -1 from me OK, and now that you've said all that, -1 from me as well; I think the idiom you describe is very common. That's why I like to discuss things on the list before filing collector items :-) I kinda wish the RAMCache manager allowed other things than REQUEST variables for differentiation. Arbitrary TALES expressions would be nice. Then we could get rid of this hard-coded gunk and get the same effect by having context/getPhysicalPath as one of the expressions. But of course then you'd have the opportunity to kill your cache performance by making the expressions too expensive. Ah well, TANSTAAFL and all that. I'll probably just drop the idea. > > For a related annoyance, see: > > http://www.zope.org/Collectors/CMF/343 > > This annoyance is indeed related, but the proper fix is for > FSPythonScript to have a ZCacheable_manage page that takes into account > the fact that it's usually part of a portal_skins setup and deal with > it. Alternativelly, portal_skins should provide the functionality of > expiring RAMCached subitems. Of those, I'd prefer the latter approach. Added this as a comment to #343. Thanks for the valuable input! -- Paul Winkler http://www.slinkp.com ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RAMcache and container vs. context
Sorry for comming late into the fray :-) Em Qui, 2005-04-21 às 19:46 -0400, Paul Winkler escreveu: > On Fri, Apr 22, 2005 at 12:27:18AM +0200, Stefan H. Holek wrote: > > Note that aq_parent() gives you the URL parent, not the container. I > > see no way around that as the return value of a script may well depend > > on its context. > > Yes, it may, agreed. Thanks much for pointing out the relevant > code, at least now I understand what's happening. > > But I still would like to argue against this behavior: > There *is* an easy alternative, and that's to put one or more > of the many location-related request variables into the > cache manager's configuration. This alternative won't help you when: * the pythonscript is sensitive to the context AND * it is called w/ 2 or more different contexts on the same request An artificial example I can come up w/ from the top of my head: Suppose you have a folder w/ an index_html that displays information about it's subfolders that is calculated by an expensive script and you want to cache the results of this script, you'd go: Folder contains shruberries. Then you'd go and RAMCache "expensively_calculate_shruberries" But if we implement the change you're suggesting, then this page would list the same number of shruberries of the first subfolder for all of them. -1 from me > For a related annoyance, see: > http://www.zope.org/Collectors/CMF/343 This annoyance is indeed related, but the proper fix is for FSPythonScript to have a ZCacheable_manage page that takes into account the fact that it's usually part of a portal_skins setup and deal with it. Alternativelly, portal_skins should provide the functionality of expiring RAMCached subitems. -- Leonardo Rochael Almeida <[EMAIL PROTECTED]> ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RAMcache and container vs. context
On Fri, Apr 22, 2005 at 12:27:18AM +0200, Stefan H. Holek wrote: > This is due to how Python Scripts compute their cache keys. The > relevant snippet from PythonScript._exec() is: > > asgns = self.getBindingAssignments() > name_context = asgns.getAssignedName('name_context', None) > if name_context: > keyset[name_context] = aq_parent(self).getPhysicalPath() > > Note that aq_parent() gives you the URL parent, not the container. I > see no way around that as the return value of a script may well depend > on its context. Yes, it may, agreed. Thanks much for pointing out the relevant code, at least now I understand what's happening. But I still would like to argue against this behavior: There *is* an easy alternative, and that's to put one or more of the many location-related request variables into the cache manager's configuration. This has two advantages: it's more explicit, and control is given to the site admin rather than hardcoding it. The disadvantages I see to using my current "container" workaround are: * The templates are client code. Having to change potentially multiple templates just to affect cacheing behavior of one script violates the Don't Repeat Yourself principle. * It's too surprising. When reviewing code, the difference between "context" and "container" brings many things to mind. Cache hit/miss ratio is not one of them. When I stumbled on the workaround, if I hadn't bothered to ask here but had merely committed the change to my templates and moved on, I would have been "programming by coincidence" (see "The Pragmatic Programmer"). I'll probably remember it now, but it seems like kind of an obscure guru-level thing for a template author to have to know, especially if they are not even a developer and only know how to write some simple TALES expressions. * It's an implicit implementation detail. The difference in caching behavior between "context" and "container" is AFAIK not documented anywhere, not even in the source code (unless you argue that the semantics of the lines you posted above count as documentation). Anyway, I have no guarantee that future versions of zope won't silently break it. * There are other reasons I might be forced to use "context". For example, if my site offers two CMF skins, each of which provides some scripts with the same names, I might have common templates that need to acquire the script from the current skin and thus may not be able to find it via "container". admittedly there are hacky workarounds in such situations, e.g. have trivial wrapper scripts live near the templates. For a related annoyance, see: http://www.zope.org/Collectors/CMF/343 -- Paul Winkler http://www.slinkp.com ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RAMcache and container vs. context
This is due to how Python Scripts compute their cache keys. The relevant snippet from PythonScript._exec() is: asgns = self.getBindingAssignments() name_context = asgns.getAssignedName('name_context', None) if name_context: keyset[name_context] = aq_parent(self).getPhysicalPath() Note that aq_parent() gives you the URL parent, not the container. I see no way around that as the return value of a script may well depend on its context. HTH, Stefan On 21. Apr 2005, at 22:26, Paul Winkler wrote: It's not obvious to me from the code why this is happening (or why it might be desirable). The cache manager stores and looks up items by their getPhysicalPath(), so I don't see why context vs. container is relevant. If I'd wanted the URL to be relevant, I would have put that in the REQUEST Variables configuration. -- Software Engineering is Programming when you can't. --E. W. Dijkstra ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] RAMcache and container vs. context
I never noticed this before today, but apparently RAMCacheManager is sensitive to whether cached items are acquired from the context or from the container. (Zope 2.7.3 but afaict it should behave the same in any recent version). Is that intended behavior or is this a bug? Let's say I have a CMF site at http://foo:8080/TheSite. And a template at /TheSite/portal_skins/custom/template1 which looks in part like like: ... Let's say that expensive_script is also in portal_skins/custom and is associated with a RAMCacheManager. Let's say that there are *no* request variables configured for this RAMCacheManager, which would seem to indicate that we consider all calls to expensive_script equivalent regardless of the request. If I visit http://foo:8080/TheSite/page1, which page makes use of this template, and I reload several times, and look at the RAMCacheManager statistics view, I see several hits for /TheSite/expensive_script. Every time I reload, the hit count goes up by one. So far so good. But if I then visit another page, http://foo:8080/TheSite/page2, which uses the same template, I get a miss and the entry count goes up by one. Reloading then gives me a hit, presumably from the new entry. If I browse around the site, the entry count increments for every new page I hit. Remember that I have nothing configured in the "REQUEST Variables" field. So the script should be the same regardless of the request right? This was puzzling me until, on a hunch, I looked at the calling template and changed it from this: ... ...to this: ... And presto, I now get only one entry, and every page on the site gives me a hit from that one entry. Is this really intended behavior? It's not obvious to me from the code why this is happening (or why it might be desirable). The cache manager stores and looks up items by their getPhysicalPath(), so I don't see why context vs. container is relevant. If I'd wanted the URL to be relevant, I would have put that in the REQUEST Variables configuration. This could be a performance hit for the unwary ... aside from all the unnecessary misses, we'd be using RAM on the order of N * M * S where N is number of cached items and M is number of pages in the site, and S is average size of return value. We only should be using N * S. -- Paul Winkler http://www.slinkp.com ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )