We have been having the problem with memory usage creeping slowly upward that I've seen reported by several others in this list. I read several of the threads on this topic in the archives but, though there were a lot of theories floated, no firm conclusions seemed to be drawn. Like the others, memory usage as reported by 'free' on the '-/+ buffers/cache' line creeps steadily upward until the site crashes. Restarting Zope frees most of the memory.
We're running RedHat Linux 6.1, on a dual- 550M processor box with 256Mb of memory, Zope 2.1.4 using Apache, mod-ssl and PCGI, ZMySqlDA 1.1.3. The ZODB holds only non-dynamic data, all user data is kept in the MySQL data base.
I tried testing on our development server while no one else was accessing the site with the 'free' utility displaying memory use every 5 seconds in an attempt to find out what activity caused memory usage to creep upward without coming back down.
To cut to the chase, I seem to have narrowed it down to access to the MySQL database from multiple threads. As it happens, we have a very thin client piece that can access the server through http outside the browser. Most use of the site is through a browser, but this small client checks with the server occasionally for scheduled tasks when the browser isn't up. In some cases, access through the browser will use a plug-in to trigger the client to check immediately rather than waiting for its infrequent poll interval.
After an afternoon of testing, I discovered that, with no one else accessing the site, I could access pages as fast as I was able through the browser only (no client) or from the client only (sped up to poll every few seconds) with no memory creep. But when both were running at once (as they normally do), the memory usage would slowly creep up.
Letting the client poll and query the MySQL database frequently while using the browser to access only pages that did NOT access the DB resulted in no creep.
Having two people simultaneously access pages that DID access the database without the client running on either machine DID cause memory creep.
Having two people access pages that did NOT access the database without the client running did NOT cause memory creep.
Looks to me like there is a memory leak when ZMySQLDA is accessed from multiple threads.
Anyone have any ideas on where I could go to pin this down further?