Re: [Zope-Dev] Zope 2.1.6 Memory issue
On Mon, 22 May 2000, Jason Spisak wrote: i did it again, and now I'm down to 160 objects total. That's more liek it. But system memory is still dropping. I was at 173632 available (This is only a half our after the Zope restart BTW. It ate 100MB in under that time.) before the second pass and now I'm at 173616. I'm *very* interested in this. I have a similar setup (stock 2.1.4 + the Lexicon.py patch, no external database, no non-bundled Products, custom CatalogAware ZClass, large number of objects (~60,000). I'm running under FreeBSD 3.4-STABLE. I tried the 'minimize' trick and got down to around 200 objects, with *no* decrease in memory used. I'm not leaking memory as fast as you, but over time memory usage does go up until swapping starts and I'm forced to restart Zope. Also as (apparently) with you, manage_debug does *not* give any clues as to what is taking the memory; everything in the list looks normal and does not add up to the observed memory usage as far as I can guess. I'm glad to see someone else is observing this memory weirdness, I thought I was the only one. --RDM ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-Dev] Zope 2.1.6 Memory issue
Shane Hathaway writes: Jason Spisak wrote: Shane Hathaway writes: Jason Spisak wrote: You must have a large site. 300,000 + objects Whoa... ;-) Tell me, what are you doing with 300K objects? Shane Technical recruiting. I am the ultimate test case for ZDOB scalability. ;) I just recently started having problems thoug, so I am going back and disabling the small fixes I have made recently in my code. I just restarted Zope again. Most of the objects just sit around. Wow. There it goes again. It's got to be DateTime.DateTime.DateTime. It just shot to 118003. That's actually a natural thing to happen. DateTime objects get attached to many kinds of objects. The real problem is that so many objects are being loaded into memory. I just did some research and it looks as though garbage collection is not fully implemented in Python. I was not aware of this; do you know whether it has been corrected? No, I had no idea. After minimizing your in-memory cache, look at the manage_debug screen again. Are the refcounts still high? I bet they are... No. The ref counts drop to hundreds. Also, is see reference to 161 Products.GadflyDA.gadfly.kjparser.ParseRule, and I'm not even useing Gadfly! What the heck is that about? Anyone? Thing's seem to be a bit nicer now that I took out an External Method that returns a DateTime object the Catalog for Indexing. However, there's only about 14,000 of objects with that property in existsnce in my DB, and they don't change often. Here's the code: def calendar_day(self) return int(self.calendar_date)/100 Used to return the unique integer day. All my best, Jason Spisak CIO HireTechs.com 6151 West Century Boulevard Suite 900 Los Angeles, CA 90045 P. 310.665.3444 F. 310.665.3544 Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-Dev] Zope 2.1.6 Memory issue
Shane Hathaway writes: Well, I reduced the 'target size' from 20,000 object down to 1000, and the memory seems to be climbing. Jason Spisak wrote: After minimizing your in-memory cache, look at the manage_debug screen again. Are the refcounts still high? I bet they are... No. The ref counts drop to hundreds. That's a big relief. Okay, now we know that this isn't a memory leak, it's only related to the fact that your site requires ZODB to load a very large number of objects into memory in order to perform normal operations. Here's something to think about: whenever an object is added to or removed from a folder, all objects in all neighboring folders and below are traversed. The manage_beforeDelete and manage_afterAdd hooks are called on each object. I am hoping to use ZPatterns to stem this rapidly out of control, ObjectManger not scalable, issue. Also, there is still the issue that Python doesn't seem to be releasing memory back to the system. I wonder whether that capability has been implemented in Python. It would really put a damper on things if is wasn't. However, when I stopped my Zope process each time, it didn't seem to give me back what Ihad started with. Maybe there is a leak in the kernel I am using. It's 2.2.12-smp. Anyone? Also, is see reference to 161 Products.GadflyDA.gadfly.kjparser.ParseRule, and I'm not even useing Gadfly! What the heck is that about? Anyone? Chances are some other product is "reaching in" to Gadfly. Do you have the Zope tutorial installed? It uses Gadfly. No. This is a Data.fs that came from Zope 1. I didn't even think to get Gadfly out of the Products directory. Thing's seem to be a bit nicer now that I took out an External Method that returns a DateTime object the Catalog for Indexing. However, there's only about 14,000 of objects with that property in existsnce in my DB, and they don't change often. Here's the code: def calendar_day(self) return int(self.calendar_date)/100 Used to return the unique integer day. That code actually doesn't return a DateTime instance, just an integer. Yeah, I didn't really think is was that. ;) Keep me informed. You bet. And a thousand thanks are waiting in my cache for you. Jason Spisak CIO HireTechs.com 6151 West Century Boulevard Suite 900 Los Angeles, CA 90045 P. 310.665.3444 F. 310.665.3544 Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-Dev] Zope 2.1.6 Memory issue
Shane Hathaway writes: Okay, here's the output. (BTW is was trying manage_debug without the Control_Panel. That was not intelligent;) Class May 22, 2000 4:21 amMay 22, 2000 4:22 amDelta OIBTree.Bucket 71328135+1003 IOBTree.BTree 25702827+257 BTree.BTree 10281105+77 IIBTree.Bucket 45904619+29 DocumentTemplate.DT_With.With 273 279 +6 2nd pass: Actually there's too damn much. Shane, I'll just send you the html. If anyone else wants to help let me know. Jason, Try /Control_Panel/manage_debug ... it will tell you exactly which kinds of objects are eating memory. Take a "snapshot" then refresh the page after a couple of minutes, and the table should list at the top the name of the class that's taking up more than its fair share. Shane Jason Spisak wrote: Zopists, I am spoiled. I have a large server and until now I have never really monitored memory usage. But I noticed things slowing down a great deal and was curious. Swapping. Machine: RH 6.1 2.2.12-smp Dual Xenon 1 gig memory Zope install: Zope 2.1 Binary ZODB pure. No DA's Home grown ZClasses. 'ps' tells me Zope is the culprit. It's using 64% when things get swappy. There are no prosesses running on this machine BTW. It's just a Zope server. My Data.fs file is ~300 packed and gets to be about 1GB at the end of every day. (A lot of transactions :) Zope is eating memory at the rate of 10MB every 2 minutes. Is that appropriate? I never noticed a slowdown until I switched to 2.1, really. I've considered writing a small python progam to compare the output of 'free' to the Zope log, but I'd love to see if there is another way to nail what's eating mem. If someone can tell me the best way to narrow it down from experience, I'll do it this afternoon. Hopefully, -- Jason Spisak [EMAIL PROTECTED] ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope ) Jason Spisak CIO HireTechs.com 6151 West Century Boulevard Suite 900 Los Angeles, CA 90045 P. 310.665.3444 F. 310.665.3544 Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. Title: Debug Information Zope Contents Undo Control Panel at /Control_Panel Debug Information Zope version: Zope 2.1.1 (binary release, python 1.5.2, linux2-x86) Python version: 1.5.2 (#10, Dec 6 1999, 12:16:27) [GCC 2.7.2.3] System Platform: linux2 Process ID: 2159 (5126) Running for: 1 hour 14 min 57 sec Top Refcounts: DateTime.DateTime.DateTime: 20559intSet.intSet: 12586OIBTree.Bucket: 8135IOBTree.Bucket: 5425BTree.Bucket: 5270IIBTree.Bucket: 4619IOBTree.BTree: 2827DocumentTemplate.DT_Util.Eval: 2168OIBTree.BTree: 1647OFS.DTMLMethod.DTMLMethod: 1548BTree.BTree: 1105ZClasses.Method.MW: 856ExtensionClass: 518DocumentTemplate.DT_In.InClass: 435ZClasses.ZClass.PersistentClass: 302DocumentTemplate.DT_With.With: 279AccessControl.User.User: 185OFS.Image.Image: 173Products.ZGadflyDA.gadfly.kjParser.ParseRule: 161DocumentTemplate.DT_Var.Var: 157ZClasses.Property.ZInstanceSheet: 135App.special_dtml.HTMLFi! le: 133ZClasses.ZClass.ZClass: 130OFS.SimpleItem.SimpleItem: 130ZClasses.Property.ZInstanceSheets: 129AccessControl.PermissionRole.PermissionRole: 127App.Factory.Factory: 123App.Permission.Permission: 118App.PersistentExtra.Persistent: 108DocumentTemplate.VSEval.Eval: 96OFS.Folder.Folder: 93App.ImageFile.ImageFile: 89SearchIndex.UnIndex.UnIndex: 68App.Product.Product: 65Acquisition.Acquirer: 57AccessControl.PermissionMapping.PM: 52ZClasses.ZClass.ZClassSheets: 50TreeDisplay.TreeTag.Tree: 50SearchIndex.UnTextIndex.UnTextIndex: 47Products.ZCatalog.CatalogAwareness.CatalogAware: 47re.RegexObject: 46Products.ExternalMethod.ExternalMethod.ExternalM! ethod: 42OFS.Image.File: 42Base: 41ZClasses.ZClass.ZObject: 37ZClasses.ObjectManager.ObjectManager: 37ts_regex.compile: 36ZServer.medusa.counter.counter: 36ZServer.PubCore.ZServerPublisher.ZServerPublisher: 35Missing.Missing: 35OFS.DTMLDocument.DTMLDocument: 34Acquisition.ImplicitAcquirerWrapper: 32DocumentTemplate.cDocumentTemplate.TemplateDict:
Re: [Zope-Dev] Zope 2.1.6 Memory issue
R. David Murray: Since this is a very heavily used Zope install, I am getting trashed faster than you can say 'leakage'. On Mon, 22 May 2000, Jason Spisak wrote: i did it again, and now I'm down to 160 objects total. That's more liek it. But system memory is still dropping. I was at 173632 available (This is only a half our after the Zope restart BTW. It ate 100MB in under that time.) before the second pass and now I'm at 173616. I'm *very* interested in this. I have a similar setup (stock 2.1.4 + the Lexicon.py patch, no external database, no non-bundled Products, custom CatalogAware ZClass, large number of objects (~60,000). I'm running under FreeBSD 3.4-STABLE. I tried the 'minimize' trick and got down to around 200 objects, with *no* decrease in memory used. I'm not leaking memory as fast as you, but over time memory usage does go up until swapping starts and I'm forced to restart Zope. Also as (apparently) with you, manage_debug does *not* give any clues as to what is taking the memory; everything in the list looks normal and does not add up to the observed memory usage as far as I can guess. I'm glad to see someone else is observing this memory weirdness, I thought I was the only one. --RDM I'm anxious to discover what's going on too. It's will really be a boon for DC to have Zope scalable to this size of ZODB and have the stability I've come to rely on. All my best, Jason Spisak CIO HireTechs.com 6151 West Century Boulevard Suite 900 Los Angeles, CA 90045 P. 310.665.3444 F. 310.665.3544 Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )