Good day everyone! I hope your weekend was as good as mine. I didn't have to work and I got to watch my son's first soccer (football) game!
For the past couple of years (I think) the Session Data Container has been leaking memory. During this time several leaks have been patch but at least for our servers these fixes have only slowed down the process. As our use of Zope grows and traffic to our site increases so does the memory usage of Zope. Unless Zope is restarted it will eventually use all physical memory and swap space and crash the server. Over the past month the problem has once again grown to the point where we must restarted Zope daily, sometimes even every 8 hours. I don't know why this problem does not seem more wide spread. My guess is because our site has a large load and we make extensive use of session variables. Fortunately, I have an easy way to reproduce the problem and even better I've found a work around. My hope is that the work around will lead somebody more familiar with the innards of Zope to where the problem is located. TEST CASE To reproduce the problem I used the following: VA Linux Server w/ 1Gig Fedora Core 3 Zope 2.7.8-final, python 2.3.4, linux2 Python 2.3.4 (#1, Feb 2 2005, 12:11:53) [GCC 3.4.2 20041017 (Red Hat 3.4.2-6.fc3)] top and/or snmp to watch memory usage Step 1: In Zope I created the following script: session = context.REQUEST.SESSION s= 'Andy' * 102400; session['sessiontest1']=s The size of 's' depends purely on how fast you would like Zope to eat memory. On my server this will consume all physical memory in about 1000 calls. Step 2: Changed the session_data object timeout to 1 minute. This is just to cause the session objects to expire in 1 minute to hurry the test along. Changed the Max subobjects to 0 (unlimited). I don't know if this is needed, but it is how we have our production servers set. Step 3: Use ab (apache bench) to call the script 1000 times using 10 concurrent threads. This causes Zope to consume over 90% of the memory as one would expect. The transient object container (/temp_folder/session_data) will now show 1009 objects. I don't know where the 6-9 extra come from.? Step 4: Wait 1-2 minutes for the objects to expire. Note: Use a browser or ab to access some other resource in Zope and or the script a number of times. This is needed to "tickle" the transient object container's expiration/garbage collection system. (You've got to keep those buckets moving) As expected the transient object container now shows 0 objects. However, Zope never releases the memory. You can wait minutes, hours, days what ever all the while tickling Zope. Zope will NEVER release the memory until it is restarted. Oh, and new requests just use more memory. Zope is NOT just hanging on to it for future use. THE WORK ARROUND I created a script called onDelete Parameter List sdo, toc sdo.clear() Then in /temp_folder/session_data I set the Script to call when objects are deleted to /onDelete Now when the tests are run the same amount of memory is consumed, but it is released back to the system when the session objects expire. CONCLUSION To me it looks like data placed in the session object are not getting deleted when the session object expires. Anything placed in a session object is essentially leaked. When the onDelete script explicitly removes the contents of the session data object the memory leak stops. Comment? Questions? Fixes? ;-) Thanks! Andy Yates _______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )