Re: [Zope-Dev] Zope 2.1.6 Memory issue
Shane Hathaway writes: Okay, here's the output. (BTW is was trying manage_debug without the Control_Panel. That was not intelligent;) Class May 22, 2000 4:21 amMay 22, 2000 4:22 amDelta OIBTree.Bucket 71328135+1003 IOBTree.BTree 25702827+257 BTree.BTree 10281105+77 IIBTree.Bucket 45904619+29 DocumentTemplate.DT_With.With 273 279 +6 2nd pass: Actually there's too damn much. Shane, I'll just send you the html. If anyone else wants to help let me know. > Jason, > > Try /Control_Panel/manage_debug ... it will tell you exactly which > kinds of objects are eating memory. Take a "snapshot" then refresh the > page after a couple of minutes, and the table should list at the top > the name of the class that's taking up more than its fair share. > > Shane > > Jason Spisak wrote: > > > > Zopists, > > > > I am spoiled. I have a large server and until now I have never really > > monitored memory usage. But I noticed things slowing down a great deal > > and was curious. Swapping. > > > > Machine: > > > > RH 6.1 > > 2.2.12-smp > > Dual Xenon > > 1 gig memory > > > > Zope install: > > > > Zope 2.1 Binary > > ZODB pure. > > No DA's > > Home grown ZClasses. > > > > 'ps' tells me Zope is the culprit. It's using 64% when things get > > swappy. There are no prosesses running on this machine BTW. It's just > > a Zope server. > > > > My Data.fs file is ~300 packed and gets to be about 1GB at the end of > > every day. (A lot of transactions :) > > > > Zope is eating memory at the rate of 10MB every 2 minutes. Is that > > appropriate? > > > > I never noticed a slowdown until I switched to 2.1, really. > > > > I've considered writing a small python progam to compare the output of > > 'free' to the Zope log, but I'd love to see if there is another way to > > nail what's eating mem. If someone can tell me the best way to narrow > > it down from experience, I'll do it this afternoon. > > > > Hopefully, > > > > -- > > Jason Spisak > > [EMAIL PROTECTED] > > > > ___ > > Zope-Dev maillist - [EMAIL PROTECTED] > > http://lists.zope.org/mailman/listinfo/zope-dev > > ** No cross posts or HTML encoding! ** > > (Related lists - > > http://lists.zope.org/mailman/listinfo/zope-announce > > http://lists.zope.org/mailman/listinfo/zope ) Jason Spisak CIO HireTechs.com 6151 West Century Boulevard Suite 900 Los Angeles, CA 90045 P. 310.665.3444 F. 310.665.3544 Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. Title: Debug Information Zope Contents Undo Control Panel at /Control_Panel Debug Information Zope version: Zope 2.1.1 (binary release, python 1.5.2, linux2-x86) Python version: 1.5.2 (#10, Dec 6 1999, 12:16:27) [GCC 2.7.2.3] System Platform: linux2 Process ID: 2159 (5126) Running for: 1 hour 14 min 57 sec Top Refcounts: DateTime.DateTime.DateTime: 20559intSet.intSet: 12586OIBTree.Bucket: 8135IOBTree.Bucket: 5425BTree.Bucket: 5270IIBTree.Bucket: 4619IOBTree.BTree: 2827DocumentTemplate.DT_Util.Eval: 2168OIBTree.BTree: 1647OFS.DTMLMethod.DTMLMethod: 1548BTree.BTree: 1105ZClasses.Method.MW: 856ExtensionClass: 518DocumentTemplate.DT_In.InClass: 435ZClasses.ZClass.PersistentClass: 302DocumentTemplate.DT_With.With: 279AccessControl.User.User: 185OFS.Image.Image: 173Products.ZGadflyDA.gadfly.kjParser.ParseRule: 161DocumentTemplate.DT_Var.Var: 157ZClasses.Property.ZInstanceSheet: 135App.special_dtml.HTMLFi! le: 133ZClasses.ZClass.ZClass: 130OFS.SimpleItem.SimpleItem: 130ZClasses.Property.ZInstanceSheets: 129AccessControl.PermissionRole.PermissionRole: 127App.Factory.Factory: 123App.Permission.Permission: 118App.PersistentExtra.Persistent: 108DocumentTemplate.VSEval.Eval: 96OFS.Folder.Folder: 93App.ImageFile.ImageFile: 89SearchIndex.UnIndex.UnIndex: 68App.Product.Product: 65Acquisition.Acquirer: 57AccessControl.PermissionMapping.PM: 52ZClasses.ZClass.ZClassSheets: 50TreeDisplay.TreeTag.Tree: 50SearchIndex.UnTextIndex.UnTextIndex: 47Products.ZCatalog.CatalogAwareness.CatalogAware: 47re.RegexObject: 46Products.ExternalMethod.ExternalMethod.ExternalM! ethod: 42OFS.Image.File: 42Base: 41ZClasses.ZClass.ZObject: 37ZClasses.ObjectManager.ObjectManager: 37ts_regex.compile: 36ZServer.medusa.counter.counter: 36ZServer.PubCore.ZServerPublisher.ZServerPublisher: 35Missing.Missing: 35OFS.DT
Re: [Zope-Dev] Zope 2.1.6 Memory issue
Kevin Dangoor: > > Here's something to think about: whenever an object is added to or > > removed from a folder, all objects in all neighboring folders and below > > are traversed. The manage_beforeDelete and manage_afterAdd hooks are > > called on each object. > > Jason, I don't know if you've taken much of a look at ZPatterns yet... Yes, I've been eyeing it and playing with it. It's been changing a bit, and I wanted to let Phillip and Ty settle it down before I moved my major production server to it. > Depending on how your code is set up, it may actually not be that difficult > for you to move some of those big Folders into Racks in Specialists. Phillip > mentioned the other day that this could help, and I think I spotted a BTree > in the Rack code yesterday, which would explain how Racks help. I'm not sure > where the rumored BTreeFolder code stands, which would be a more immediate > drop in fix. > > It didn't look to me like Specialists were too difficult to work with, > though. > > Kevin > That's really where I want to go. I'm sure Phillip would love to have a big fat example of ZPatterns getting hammered and working to show off too. :) All my best, Jason Spisak CIO HireTechs.com 6151 West Century Boulevard Suite 900 Los Angeles, CA 90045 P. 310.665.3444 F. 310.665.3544 Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-Dev] Zope 2.1.6 Memory issue
On Mon, 22 May 2000, Jason Spisak wrote: > > > Also, is see reference to 161 Products.GadflyDA.gadfly.kjparser.ParseRule, > > > and I'm not even useing Gadfly! [...] > > Chances are some other product is "reaching in" to Gadfly. Do you have > > the Zope tutorial installed? It uses Gadfly. > > No. This is a Data.fs that came from Zope 1. I didn't even think to get > Gadfly out of the Products directory. I've seen these gadfly references ever since I started looking at the manage_debug screen. My Data.fs also has no extra products installed. If something is reaching into Gadfly, it is something that ships with Zope. By the way, my site is structured into a series of folders, so that no folder has more than about 50 objects in it, with the exception (of course ) of the most heavily modified folder, which has about 120 objects. But I'm still seeing this growing-memory-usage pattern. However, the memory usage does grow pretty much just when a large number of objects get updated, usually in many different folders. So if there is a Python garbage collection issue, I think that would probably explain my case, at least. It seems incredible that Python garbage collection would be broken, but I suppose Zope *is* one of the most intensive programs running in python, so if anyone is going to see it it is going to be our community. --RDM ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-Dev] Zope 2.1.6 Memory issue
- Original Message - From: "Shane Hathaway" <[EMAIL PROTECTED]> To: "Jason Spisak" <[EMAIL PROTECTED]> Cc: <[EMAIL PROTECTED]> Sent: Monday, May 22, 2000 6:29 PM Subject: Re: [Zope-Dev] Zope 2.1.6 Memory issue > Here's something to think about: whenever an object is added to or > removed from a folder, all objects in all neighboring folders and below > are traversed. The manage_beforeDelete and manage_afterAdd hooks are > called on each object. Jason, I don't know if you've taken much of a look at ZPatterns yet... Depending on how your code is set up, it may actually not be that difficult for you to move some of those big Folders into Racks in Specialists. Phillip mentioned the other day that this could help, and I think I spotted a BTree in the Rack code yesterday, which would explain how Racks help. I'm not sure where the rumored BTreeFolder code stands, which would be a more immediate drop in fix. It didn't look to me like Specialists were too difficult to work with, though. Kevin ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-Dev] Zope 2.1.6 Memory issue
Shane Hathaway writes: Well, I reduced the 'target size' from 20,000 object down to 1000, and the memory seems to be climbing. > Jason Spisak wrote: > > > > > > After minimizing your in-memory cache, look at the manage_debug screen > > > again. Are the refcounts still high? I bet they are... > > > > > > > No. The ref counts drop to hundreds. > > That's a big relief. Okay, now we know that this isn't a memory leak, > it's only related to the fact that your site requires ZODB to load a > very large number of objects into memory in order to perform normal > operations. > > Here's something to think about: whenever an object is added to or > removed from a folder, all objects in all neighboring folders and below > are traversed. The manage_beforeDelete and manage_afterAdd hooks are > called on each object. I am hoping to use ZPatterns to stem this rapidly out of control, ObjectManger not scalable, issue. > > Also, there is still the issue that Python doesn't seem to be releasing > memory back to the system. I wonder whether that capability has been > implemented in Python. > It would really put a damper on things if is wasn't. However, when I stopped my Zope process each time, it didn't seem to give me back what Ihad started with. Maybe there is a leak in the kernel I am using. It's 2.2.12-smp. Anyone? > > Also, is see reference to 161 Products.GadflyDA.gadfly.kjparser.ParseRule, > > and I'm not even useing Gadfly! > > > > What the heck is that about? Anyone? > > Chances are some other product is "reaching in" to Gadfly. Do you have > the Zope tutorial installed? It uses Gadfly. > No. This is a Data.fs that came from Zope 1. I didn't even think to get Gadfly out of the Products directory. > > > > Thing's seem to be a bit nicer now that I took out an External Method that > > returns a DateTime object the Catalog for Indexing. However, there's only > > about 14,000 of objects with that property in existsnce in my DB, and they > > don't change often. > > > > Here's the code: > > > > def calendar_day(self) > > return int(self.calendar_date)/100 > > > > Used to return the unique integer day. > > That code actually doesn't return a DateTime instance, just an integer. > Yeah, I didn't really think is was that. ;) > Keep me informed. You bet. And a thousand thanks are waiting in my cache for you. Jason Spisak CIO HireTechs.com 6151 West Century Boulevard Suite 900 Los Angeles, CA 90045 P. 310.665.3444 F. 310.665.3544 Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-Dev] Zope 2.1.6 Memory issue
Jason Spisak wrote: > > > > After minimizing your in-memory cache, look at the manage_debug screen > > again. Are the refcounts still high? I bet they are... > > > > No. The ref counts drop to hundreds. That's a big relief. Okay, now we know that this isn't a memory leak, it's only related to the fact that your site requires ZODB to load a very large number of objects into memory in order to perform normal operations. Here's something to think about: whenever an object is added to or removed from a folder, all objects in all neighboring folders and below are traversed. The manage_beforeDelete and manage_afterAdd hooks are called on each object. Also, there is still the issue that Python doesn't seem to be releasing memory back to the system. I wonder whether that capability has been implemented in Python. > Also, is see reference to 161 Products.GadflyDA.gadfly.kjparser.ParseRule, > and I'm not even useing Gadfly! > > What the heck is that about? Anyone? Chances are some other product is "reaching in" to Gadfly. Do you have the Zope tutorial installed? It uses Gadfly. > > Thing's seem to be a bit nicer now that I took out an External Method that > returns a DateTime object the Catalog for Indexing. However, there's only > about 14,000 of objects with that property in existsnce in my DB, and they > don't change often. > > Here's the code: > > def calendar_day(self) > return int(self.calendar_date)/100 > > Used to return the unique integer day. That code actually doesn't return a DateTime instance, just an integer. Keep me informed. Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-Dev] Zope 2.1.6 Memory issue
Shane Hathaway writes: > Jason Spisak wrote: > > > > Shane Hathaway writes: > > > > > Jason Spisak wrote: > > > > > You must have a large site. > > > > > > > > 300,000 + objects > > > > > > Whoa... ;-) > > > > > > Tell me, what are you doing with 300K objects? > > > > > > Shane > > > > Technical recruiting. > > I am the ultimate test case for ZDOB scalability. ;) > > > > I just recently started having problems thoug, so I am going back and > > disabling the small fixes I have made recently in my code. > > > > I just restarted Zope again. > > > > Most of the objects just sit around. > > > > Wow. There it goes again. It's got to be DateTime.DateTime.DateTime. It > > just shot to 118003. > > That's actually a natural thing to happen. DateTime objects get > attached to many kinds of objects. The real problem is that so many > objects are being loaded into memory. > > I just did some research and it looks as though garbage collection is > not fully implemented in Python. I was not aware of this; do you know > whether it has been corrected? No, I had no idea. > > After minimizing your in-memory cache, look at the manage_debug screen > again. Are the refcounts still high? I bet they are... > No. The ref counts drop to hundreds. Also, is see reference to 161 Products.GadflyDA.gadfly.kjparser.ParseRule, and I'm not even useing Gadfly! What the heck is that about? Anyone? Thing's seem to be a bit nicer now that I took out an External Method that returns a DateTime object the Catalog for Indexing. However, there's only about 14,000 of objects with that property in existsnce in my DB, and they don't change often. Here's the code: def calendar_day(self) return int(self.calendar_date)/100 Used to return the unique integer day. All my best, Jason Spisak CIO HireTechs.com 6151 West Century Boulevard Suite 900 Los Angeles, CA 90045 P. 310.665.3444 F. 310.665.3544 Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-Dev] Zope 2.1.6 Memory issue
Shane Hathaway writes: > Jason Spisak wrote: > > > You must have a large site. > > > > 300,000 + objects > > Whoa... ;-) > > Tell me, what are you doing with 300K objects? > > Shane After being up for about a half-hour my objects reading are: Total number of objects in the database 300280 Total number of objects in all of the caches combined 150995 Is this typical? Half? I doubt it. My target size is about 20,000. All my best, Jason Spisak CIO HireTechs.com 6151 West Century Boulevard Suite 900 Los Angeles, CA 90045 P. 310.665.3444 F. 310.665.3544 Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-Dev] Zope 2.1.6 Memory issue
R. David Murray: Since this is a very heavily used Zope install, I am getting trashed faster than you can say 'leakage'. > On Mon, 22 May 2000, Jason Spisak wrote: > > i did it again, and now I'm down to 160 objects total. > > That's more liek it. > > But system memory is still dropping. I was at 173632 available (This is > > only a half our after the Zope restart BTW. It ate 100MB in under that > > time.) before the second pass and now I'm at 173616. > > I'm *very* interested in this. > > I have a similar setup (stock 2.1.4 + the Lexicon.py patch, no > external database, no non-bundled Products, custom CatalogAware > ZClass, large number of objects (~60,000). I'm running under > FreeBSD 3.4-STABLE. > > I tried the 'minimize' trick and got down to around 200 objects, > with *no* decrease in memory used. I'm not leaking memory as fast > as you, but over time memory usage does go up until swapping starts > and I'm forced to restart Zope. > > Also as (apparently) with you, manage_debug does *not* give > any clues as to what is taking the memory; everything in the list > looks normal and does not add up to the observed memory usage > as far as I can guess. > > I'm glad to see someone else is observing this memory weirdness, > I thought I was the only one. > > --RDM > I'm anxious to discover what's going on too. It's will really be a boon for DC to have Zope scalable to this size of ZODB and have the stability I've come to rely on. All my best, Jason Spisak CIO HireTechs.com 6151 West Century Boulevard Suite 900 Los Angeles, CA 90045 P. 310.665.3444 F. 310.665.3544 Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-Dev] Zope 2.1.6 Memory issue
On Mon, 22 May 2000, Jason Spisak wrote: > i did it again, and now I'm down to 160 objects total. > That's more liek it. > But system memory is still dropping. I was at 173632 available (This is > only a half our after the Zope restart BTW. It ate 100MB in under that > time.) before the second pass and now I'm at 173616. I'm *very* interested in this. I have a similar setup (stock 2.1.4 + the Lexicon.py patch, no external database, no non-bundled Products, custom CatalogAware ZClass, large number of objects (~60,000). I'm running under FreeBSD 3.4-STABLE. I tried the 'minimize' trick and got down to around 200 objects, with *no* decrease in memory used. I'm not leaking memory as fast as you, but over time memory usage does go up until swapping starts and I'm forced to restart Zope. Also as (apparently) with you, manage_debug does *not* give any clues as to what is taking the memory; everything in the list looks normal and does not add up to the observed memory usage as far as I can guess. I'm glad to see someone else is observing this memory weirdness, I thought I was the only one. --RDM ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-Dev] Zope 2.1.6 Memory issue
Shane Hathaway writes: > Go to Control Panel -> Database Management -> Flush Cache and click > "minimize". Then go to Control Panel -> Database Management -> Cache > Parameters. After a minimize operation, the "Total number of objects > in all of the caches combined" should be relatively low (< 1000), as > well as your system memory usage. Is that the case? > i did it again, and now I'm down to 160 objects total. That's more liek it. But system memory is still dropping. I was at 173632 available (This is only a half our after the Zope restart BTW. It ate 100MB in under that time.) before the second pass and now I'm at 173616. Jason Spisak CIO HireTechs.com 6151 West Century Boulevard Suite 900 Los Angeles, CA 90045 P. 310.665.3444 F. 310.665.3544 Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-Dev] Zope 2.1.6 Memory issue
Shane Hathaway writes: > Jason Spisak wrote: > > > > Shane Hathaway writes: > > > > I had to restart Zope. I was about to start swapping, and now everyone's a > > lunch. > > > > It's so weird. Even though there is really no one on it that memory is > > just flying out the window. > > > > > Jason Spisak wrote: > > > > > > > > Shane Hathaway writes: > > > > > > > > Okay, here's the output. (BTW is was trying manage_debug without the > > > > Control_Panel. That was not intelligent;) > > > > > > > > Class May 22, 2000 4:21 amMay 22, 2000 4:22 amDelta > > > > OIBTree.Bucket 71328135+1003 > > > > IOBTree.BTree 25702827+257 > > > > BTree.BTree 10281105+77 > > > > IIBTree.Bucket 45904619+29 > > > > > > Since you say you are not using any DA's, the culprit must be > > > ZCatalog. What happens when you minimize the memory cache (setting the > > > limit to 3 seconds)? Do the OIBTree.Bucket objects get unloaded? > > > > > > Shane > > > > How would I tell if they get unloaded? I am new to the manage_debug BTW. > > I set the Cache Parameters target time between accesses to 3. What am I > > looking for number-wise? > > I studied your debug screens a little more and realized that the > handling of BTree, etc. is probably in order. The refcount is indeed > decreasing--I think you just happened to hit a spike. > > You must have a large site. 300,000 + objects >I'm guessing that you have a lot of > objects that get cataloged. Yep. That's why I have the machine I do, though. >In order for them to be cataloged, they > have to be loaded into memory then unloaded. > I see. > Go to Control Panel -> Database Management -> Flush Cache and click > "minimize". Then go to Control Panel -> Database Management -> Cache > Parameters. After a minimize operation, the "Total number of objects > in all of the caches combined" should be relatively low (< 1000), as > well as your system memory usage. Is that the case? No. After the 'minimize' I now have 15501 objects in cache combined. And the free command shows less memory than before the minimize command. As Wyle E. Coyote would say...Yipe. :o Jason Spisak CIO HireTechs.com 6151 West Century Boulevard Suite 900 Los Angeles, CA 90045 P. 310.665.3444 F. 310.665.3544 Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-Dev] Zope 2.1.6 Memory issue
Shane Hathaway writes: It doesn't look like the number of objects is dropping like it should when it's set at 3 seconds. After 30 min I'm still at DateTime.DateTime.DateTime +7398 BTree.Bucket+3501 intSet.intSet +1863 IOBTree.Bucket +1572 > Jason Spisak wrote: > > > > Shane Hathaway writes: > > > > Okay, here's the output. (BTW is was trying manage_debug without the > > Control_Panel. That was not intelligent;) > > > > Class May 22, 2000 4:21 amMay 22, 2000 4:22 amDelta > > OIBTree.Bucket 71328135+1003 > > IOBTree.BTree 25702827+257 > > BTree.BTree 10281105+77 > > IIBTree.Bucket 45904619+29 > > Since you say you are not using any DA's, the culprit must be > ZCatalog. What happens when you minimize the memory cache (setting the > limit to 3 seconds)? Do the OIBTree.Bucket objects get unloaded? > > Shane > > ___ > Zope-Dev maillist - [EMAIL PROTECTED] > http://lists.zope.org/mailman/listinfo/zope-dev > ** No cross posts or HTML encoding! ** > (Related lists - > http://lists.zope.org/mailman/listinfo/zope-announce > http://lists.zope.org/mailman/listinfo/zope ) > Jason Spisak CIO HireTechs.com 6151 West Century Boulevard Suite 900 Los Angeles, CA 90045 P. 310.665.3444 F. 310.665.3544 Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-Dev] Zope 2.1.6 Memory issue
Shane Hathaway writes: I had to restart Zope. I was about to start swapping, and now everyone's a lunch. It's so weird. Even though there is really no one on it that memory is just flying out the window. > Jason Spisak wrote: > > > > Shane Hathaway writes: > > > > Okay, here's the output. (BTW is was trying manage_debug without the > > Control_Panel. That was not intelligent;) > > > > Class May 22, 2000 4:21 amMay 22, 2000 4:22 amDelta > > OIBTree.Bucket 71328135+1003 > > IOBTree.BTree 25702827+257 > > BTree.BTree 10281105+77 > > IIBTree.Bucket 45904619+29 > > Since you say you are not using any DA's, the culprit must be > ZCatalog. What happens when you minimize the memory cache (setting the > limit to 3 seconds)? Do the OIBTree.Bucket objects get unloaded? > > Shane How would I tell if they get unloaded? I am new to the manage_debug BTW. I set the Cache Parameters target time between accesses to 3. What am I looking for number-wise? Thanks again, Jason Spisak CIO HireTechs.com 6151 West Century Boulevard Suite 900 Los Angeles, CA 90045 P. 310.665.3444 F. 310.665.3544 Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-Dev] Zope 2.1.6 Memory issue
Jason Spisak wrote: > > Shane Hathaway writes: > > Okay, here's the output. (BTW is was trying manage_debug without the > Control_Panel. That was not intelligent;) > > Class May 22, 2000 4:21 amMay 22, 2000 4:22 amDelta > OIBTree.Bucket 71328135+1003 > IOBTree.BTree 25702827+257 > BTree.BTree 10281105+77 > IIBTree.Bucket 45904619+29 Since you say you are not using any DA's, the culprit must be ZCatalog. What happens when you minimize the memory cache (setting the limit to 3 seconds)? Do the OIBTree.Bucket objects get unloaded? Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope 2.1.6 Memory issue
Jason, Try /Control_Panel/manage_debug ... it will tell you exactly which kinds of objects are eating memory. Take a "snapshot" then refresh the page after a couple of minutes, and the table should list at the top the name of the class that's taking up more than its fair share. Shane Jason Spisak wrote: > > Zopists, > > I am spoiled. I have a large server and until now I have never really > monitored memory usage. But I noticed things slowing down a great deal > and was curious. Swapping. > > Machine: > > RH 6.1 > 2.2.12-smp > Dual Xenon > 1 gig memory > > Zope install: > > Zope 2.1 Binary > ZODB pure. > No DA's > Home grown ZClasses. > > 'ps' tells me Zope is the culprit. It's using 64% when things get > swappy. There are no prosesses running on this machine BTW. It's just > a Zope server. > > My Data.fs file is ~300 packed and gets to be about 1GB at the end of > every day. (A lot of transactions :) > > Zope is eating memory at the rate of 10MB every 2 minutes. Is that > appropriate? > > I never noticed a slowdown until I switched to 2.1, really. > > I've considered writing a small python progam to compare the output of > 'free' to the Zope log, but I'd love to see if there is another way to > nail what's eating mem. If someone can tell me the best way to narrow > it down from experience, I'll do it this afternoon. > > Hopefully, > > -- > Jason Spisak > [EMAIL PROTECTED] > > ___ > Zope-Dev maillist - [EMAIL PROTECTED] > http://lists.zope.org/mailman/listinfo/zope-dev > ** No cross posts or HTML encoding! ** > (Related lists - > http://lists.zope.org/mailman/listinfo/zope-announce > http://lists.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
