On Sat, 2003-08-23 at 22:18, Shane Hathaway wrote:When you flush the cache, those DateTimes should disappear. If they don't, the leak is keeping them.
They are disappearing. Too bad they return immediately after, as they're comming from a very heavy catalog query result.
Hmm, that tends to indicate you really need that much memory.
1) The ZODB cache size. The meaning of this number changed dramatically in 2.6. Before 2.6 it was a very vague number. In 2.6 it's a target number of objects that Zope actually tries to maintain. Before 2.6 it might have made sense to set the ZODB cache size to some arbitrarily high number like 100,000; in 2.6 you want to start at about 2000 and adjust from there. There are tools in 2.6 for helping you adjust the number.
We're keeping it at 15,000. Most threads keep around 12k and 13k. One of them sometimes exagerates at 100k or so. Hmm, funny... when I look at them now, they're all under 5k...
Maybe the individual ZODB objects are excessively large. A few months ago we found a bug in the new zope.org software that made it store file uploads as a single chunk. A lot of those files were over 1 MB in size. This exhausted 1 GB of RAM pretty quickly. Splitting the files into chunks, like Zope usually does automatically, solved the problem.
2) The number of ImplicitAcquisitionWrappers present in the system. I have found it to be a reliable indicator of whether you have a leak or not. Expect this number to stay under 400 or so. If it grows gradually, there's a leak. Watch the refcounts screen.
It keeps under 200. At least when I'm looking :-)
That's another strong indication that it's not actually leaking. Or perhaps it's a class of leak I have not yet encountered. ;-)
4) Don't use 2.6.1. Use 2.6.2, which has fixes for known leaks. It is actually already tagged in CVS as "Zope-2_6_2", and it's what zope.org is now running. Various unrelated things prevented a formal release this week.
You guys are sure about this? The client is very much against running things from CVS because they just don't want to go thru another upgrade processes when the "real" 2.6.2 shows up.
Well, 2.6.2b6 has been released (but not announced), and I've been promised there will be no more changes before 2.6.2.
If all else fails, grep all Python modules for "sys._getframe()" and "sys.exc_info()". These are the primary causes of memory leaks in Python 2.1 and below. If you're brave, you can just run Zope under Python 2.2, which fixes those particular leaks AFAIK.
Do I need to be particularly brave to run 2.6.2 (as tagged in the CVS) under Python2.2? is it still an "unsupported" combination? I know it's 2.7's job to be 2.2 compliant, but I've seen reports of more and more people running Zope 2.6.x under Python 2.2.
2.6 is also intended to be compatible with Python 2.2. It's just that here at ZC we haven't gone through the rigorous analysis we've been planning. Hmm, come to think of it, the community could actually do that analysis. It wouldn't be terribly difficult. Any volunteers?
Finally, there's always hope. :-) The latest thing I've been doing is running Zope in a debug build of Python. A debug build makes a magical "sys._getobjects()" available,
Where do you discover these things? :-)
I learned this one from Jeremy Hylton. He posted some info about it on his weblog after he mentioned it.
allowing you to inspect all live objects through a remote console.
Interesting, which one do you use? does Boa have a remote console? or is it just plain monitor port?
I rolled my own remote console. The monitor port is too buggy and Boa is overkill for this.
Since debug builds aren't much slower than standard builds, you can even run a debug build in production for short periods of time. I've been building a small library of functions for working in this mode, and if you need them, I'll pass them along. I'd have to warn you that they are anything but intuitive in their purpose and use, though.
I'll let you know if it gets to that :-)
For now we'll be stress testing to try and locate the URLs that cause the most leakage.
That's an excellent approach.
Zope-Dev maillist - [EMAIL PROTECTED]
** No cross posts or HTML encoding! **
(Related lists - http://mail.zope.org/mailman/listinfo/zope-announce