Re: [Zope-dev] copy & paste 'leakage'
Martijn Faassen wrote: > Another data point. Copy & paste of ParsedXML documents is normal > and fast when the object is in a folder not surrounded by too > many other folders (or objects in general, not sure yet). If I create > a bunch of very large folders sitting next to the ParsedXML document > that I'm going to copy, the copying process slows down and generates > a lot of extraneous megs that I can then clean out again from the ZODB. > > Somehow the document is being affected by its environment, though I > doubt I have the full story yet (I did have problems with folders that > were not filled with anything else before). The manage_beforeDelete() and manage_afterAdd() events are propagated to subfolders, meaning that a whole lot of objects have to wake up every time you add or remove something. Or it might be manage_afterClone() being called, which is very similar. You could add a temporary print statement to ObjectManager.manage_after* to see just how often the methods are getting called. The solution? Event management. Make objects register to receive events. Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] copy & paste 'leakage'
Martijn Faassen wrote: >Hey, > >I'm running into a weird problem I'm not sure how to tackle. I've >noticed that under some circumstances it takes a long time to copy >and paste a ParsedXML object. This seems to happen in a clean Zope, >at least in the Zope root, though it doesn't seem to happen in folders. >I've also had it happen in Folderish objects in an app, though *sometimes* >the problem wouldn't occur. Quite peculiar. > >I noticed that it wasn't only slow; it was generated huge amounts of >useless volume for the ZODB as seen in the database management screen; >objects a couple of K big generally cause a meg or two of leak when >the copy process is 'slow'. > >I concluded the copy process is this slow because Zope is somehow >copying far more data than intended. > >But I don't know what; I need to get a clue in order to tackle this problem, >and this is as far as I've gotten. I've speculated somekind of weird >acquisition wrapper issues may be involved (certainly possible with ParsedXML), >which cause other objects to be copied, or possibly the same XML nodes >to be copied, multiple times, even though I can't find them. But perhaps >someone else has run into this problem before and can give me some idea >on where to start looking. > >Regards, > >Martijn > Martijn, under Zope 2.5, there's a utilites/ZODBTools/fstest.py program which can be used to take a look at your Data.fs file -- if you run it with -v -v (two verbose flags) it will print a line for each object in every transaction. If you look at the tail end of the output, you should be seeing the transactions performed last. You should be able to do some degree of differenentiation based on that to figure out the volume of objects being copied in the FileStorage. -- Matt Kromer Zope Corporation http://www.zope.com/ ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] copy & paste 'leakage'
Hi again, Another data point. Copy & paste of ParsedXML documents is normal and fast when the object is in a folder not surrounded by too many other folders (or objects in general, not sure yet). If I create a bunch of very large folders sitting next to the ParsedXML document that I'm going to copy, the copying process slows down and generates a lot of extraneous megs that I can then clean out again from the ZODB. Somehow the document is being affected by its environment, though I doubt I have the full story yet (I did have problems with folders that were not filled with anything else before). Regards, Martijn ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] copy & paste 'leakage'
Hey, I'm running into a weird problem I'm not sure how to tackle. I've noticed that under some circumstances it takes a long time to copy and paste a ParsedXML object. This seems to happen in a clean Zope, at least in the Zope root, though it doesn't seem to happen in folders. I've also had it happen in Folderish objects in an app, though *sometimes* the problem wouldn't occur. Quite peculiar. I noticed that it wasn't only slow; it was generated huge amounts of useless volume for the ZODB as seen in the database management screen; objects a couple of K big generally cause a meg or two of leak when the copy process is 'slow'. I concluded the copy process is this slow because Zope is somehow copying far more data than intended. But I don't know what; I need to get a clue in order to tackle this problem, and this is as far as I've gotten. I've speculated somekind of weird acquisition wrapper issues may be involved (certainly possible with ParsedXML), which cause other objects to be copied, or possibly the same XML nodes to be copied, multiple times, even though I can't find them. But perhaps someone else has run into this problem before and can give me some idea on where to start looking. Regards, Martijn ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
