Re: [Zope] Large jumps of memory use increase, and seeking overall understanding of memory use

2007-02-25 Thread Jens Vagelpohl
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 25 Feb 2007, at 00:05, Jean Lagarde wrote: I fully understand that Zope has to be restarted regularly This may be needed in a situation where you have a memory-hungry application (like Plone), but otherwise there is no general need to

Re: [Zope] Large jumps of memory use increase, and seeking overall understanding of memory use

2007-02-25 Thread Jens Vagelpohl
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Alex and I are volunteers doing this work for a non-profit, so any ideas or yep, that's normal, live with it would be greatly appreciated! (If the latter answer, I would still like to better understand where that memory is being used; I still don't

Re: [Zope] Large jumps of memory use increase, and seeking overall understanding of memory use

2007-02-25 Thread Jean Lagarde
On 2/25/07, Jens Vagelpohl [EMAIL PROTECTED] wrote: On 25 Feb 2007, at 00:05, Jean Lagarde wrote: I fully understand that Zope has to be restarted regularly This may be needed in a situation where you have a memory-hungry application (like Plone), but otherwise there is no general need to

Re: [Zope] Large jumps of memory use increase, and seeking overall understanding of memory use

2007-02-25 Thread Jens Vagelpohl
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 25 Feb 2007, at 18:58, Jean Lagarde wrote: Memory size was at 417640 (Kb) from 03:01:40 to 03:04:41 (for three minutes). During these three minutes, there were requests for our front page and apparently everything that accompanies it in Plone,

Re: [Zope] Large jumps of memory use increase, and seeking overall understanding of memory use

2007-02-25 Thread Maciej Wisniowski
I've not seen whole thread so sorry if I missed something. there were requests for our front page and apparently everything that accompanies it in Plone, i.e. the Plone scripts.jss, the style sheets, bunch of icon gifs, image thumbs the logo.jpg, etc. Maybe this is your specific

Re: [Zope] Large jumps of memory use increase, and seeking overall understanding of memory use

2007-02-25 Thread Jean Lagarde
On 2/25/07, Jens Vagelpohl [EMAIL PROTECTED] wrote: Comparing the number of bytes served during a request and how much memory is consumed is a bit naive. You have *no* idea what all needs to be touched internally to serve that one object. The weightier the framework you run on to of Zope, the

Re: [Zope] Large jumps of memory use increase, and seeking overall understanding of memory use

2007-02-25 Thread alex
I've not seen whole thread so sorry if I missed something. there were requests for our front page and apparently everything that accompanies it in Plone, i.e. the Plone scripts.jss, the style sheets, bunch of icon gifs, image thumbs the logo.jpg, etc. Maybe this is your specific configuration

Re: [Zope] Large jumps of memory use increase, and seeking overall understanding of memory use

2007-02-25 Thread Jens Vagelpohl
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 25 Feb 2007, at 20:07, Jean Lagarde wrote: 3) As for all the baggage that might be tied to an object in the ZODB, I will admit a lot of naivete there, but Zope's behavior should not be black magic. In the end, behavior has to be deterministic.

Re: [Zope] Large jumps of memory use increase, and seeking overall understanding of memory use

2007-02-25 Thread Jens Vagelpohl
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 25 Feb 2007, at 19:44, Maciej Wisniowski wrote: I've not seen whole thread so sorry if I missed something. there were requests for our front page and apparently everything that accompanies it in Plone, i.e. the Plone scripts.jss, the style

Re: [Zope] Large jumps of memory use increase, and seeking overall understanding of memory use

2007-02-25 Thread Jean Lagarde
Thanks Maciej, Yes, I am using CacheFu (a comprehensive caching product for Plone). I have Squid installed and ready to go (had it in the loop for a little while, but we were having all kinds of server instabilities at some point and I turned that one off to simplify things; we are more stable

Re: [Zope] Large jumps of memory use increase, and seeking overall understanding of memory use

2007-02-25 Thread Jean Lagarde
Fair enough about me blaming Zope when Plone is the likely issue (I actually meant the system as a whole, but that's my bad for not thinking and writing clearly). As for the effort required to get to the bottom of these issues, then we are in agreement; I won't take the time to do so either (at

Re: [Zope] Large jumps of memory use increase, and seeking overall understanding of memory use

2007-02-25 Thread Jean Lagarde
On 2/25/07, Jens Vagelpohl [EMAIL PROTECTED] wrote: Serving from cache still means they need to be touched at least once to get them into the cache, and (in case of normal caching behavior) they will be touched again once the cached record expires. Caching will not make any difference to memory

Re: [Zope] Large jumps of memory use increase, and seeking overall understanding of memory use

2007-02-25 Thread Maciej Wisniowski
Serving from cache still means they need to be touched at least once to get them into the cache, and (in case of normal caching behavior) they will be touched again once the cached record expires. Yes, right. I tried to write this with 'always': not served (always) directly by Zope/Plone

Re: [Zope] Large jumps of memory use increase, and seeking overall understanding of memory use

2007-02-25 Thread Maciej Wisniowski
Now I will again admit to some overall experience, with even HTTP, so correct me if I'm wrong: Without squid, I can either cache objects in a Zope memory cache or in browser caches using headers (that's part of what CacheFu helps to configure). However, neither of those prevents Zope from

Re: [Zope] Large jumps of memory use increase, and seeking overall understanding of memory use

2007-02-25 Thread Jens Vagelpohl
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 25 Feb 2007, at 20:48, Jean Lagarde wrote: On 2/25/07, Jens Vagelpohl [EMAIL PROTECTED] wrote: Serving from cache still means they need to be touched at least once to get them into the cache, and (in case of normal caching behavior) they will

Re: [Zope] Large jumps of memory use increase, and seeking overall understanding of memory use

2007-02-25 Thread Jens Vagelpohl
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 25 Feb 2007, at 21:19, Maciej Wisniowski wrote: Caching will not make any difference to memory behavior I'd say. I'm not expert here, so I might be wrong but AFAIK Zope instance loads 'touched' objects from ZODB into cache. Each ZODB connection

Re: [Zope] Large jumps of memory use increase, and seeking overall understanding of memory use

2007-02-25 Thread Maciej Wisniowski
This is a bit like clutching at straws. You're doctoring the symptoms, hoping that some things may not be loaded into memory. It's not a real solution. Memory-hungry applications will remain memory-hungry, cached or not. I'm just trying to do something, just like Jean does. You're convincing

Re: [Zope] Large jumps of memory use increase, and seeking overall understanding of memory use

2007-02-25 Thread Jens Vagelpohl
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 26 Feb 2007, at 00:41, Maciej Wisniowski wrote: One more thing. There is a setting in zope.conf that says about cache size. It is used with ZEO setups. Maybe it is worth looking at how big value it is set to. The ZEO cache is purely

[Zope] Large jumps of memory use increase, and seeking overall understanding of memory use

2007-02-24 Thread Jean Lagarde
All, (I am posting to both the general Zope and Plone lists, I hope that's all right. My questions are more directly Zope issues, but in case someone who only looks only at the Plone lists has a different insight to offer...) I'm the co-admin Alex Kirk was referring to in his recent post Random