Re: [Zope-dev] TypesTool speedup
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Alec Mitchell wrote: > Looking at the changes rocky made, I don't see why allowing external > methods from non-Products should require making the whole process > completely dynamic on every call. The old mechanism of storing the > product list should be put back in place, along with the addition of > non-Product information to these results. Not to mention that the *idea* of having ExternalMethods loaded from non-products is completely silly: it the package is written to function as a Zope2 product (and I defy anybody to offer an example of a non-Zope2-specific package with an 'Extensions' directory!), then make it a product, dangit. Tres. - -- === Tres Seaver +1 540-429-0999 [EMAIL PROTECTED] Palladion Software "Excellence by Design"http://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFF7AiH+gerLs4ltQ4RAqyBAJ4yMkUo+japLU1tTrWvDcq7skr+cgCg2Vci 3YMGrdESMpqCe8EXwxKIsFA= =DEeK -END PGP SIGNATURE- ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] TypesTool speedup
Looking at the changes rocky made, I don't see why allowing external methods from non-Products should require making the whole process completely dynamic on every call. The old mechanism of storing the product list should be put back in place, along with the addition of non-Product information to these results. Alec ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] TypesTool speedup
Previously Sidnei da Silva wrote: > Mutable default values are evil. I would get rid of that. I modeled that on how get_module_info worked, so at least the pattern is not new in the Zope2 codebase. It could just as easily be done in a global value as well. I'm more interested in seeing if people agree with the basic approach though. Wichert. -- Wichert Akkerman <[EMAIL PROTECTED]>It is simple to make things. http://www.wiggy.net/ It is hard to make things simple. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] TypesTool speedup
Mutable default values are evil. I would get rid of that. -- Sidnei da Silva Enfold Systemshttp://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214 ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] TypesTool speedup
Previously Wichert Akkerman wrote: > http://paste.plone.org/13217 should do the trick. It makes > _product_packages cache its result using the list of products in > Control_Panel as a cache key. That makes sure that removing or adding > products there will not result in stale data being returned by the > dispatcher. Obvious and stupid mistake in that patch, http://paste.plone.org/13218 should be better. Wichert. -- Wichert Akkerman <[EMAIL PROTECTED]>It is simple to make things. http://www.wiggy.net/ It is hard to make things simple. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] TypesTool speedup
Previously Andreas Jung wrote: > --On 4. März 2007 13:57:23 +0100 Wichert Akkerman <[EMAIL PROTECTED]> > wrote: > > >I'm forwarding a message from limi here, since it makes much more sense > >here than it does on plone-developers. To summarize it: while analying > >performance of the Plone 3 codebase they noticed a lot of time was spent > >trying to figure out which types could be added at a location. Part of > >that logic uses the TypesTool _queryFactoryMethod function, which goes > >through the dispatcher to get to the factory method for a type, which is > >apparently very slow in Zope 2.10, and much faster in Zope 2.9. > > > > Any idea why the code is different between 2.9 and 2.10? Rocky modified it in changeset 67869 to be able to support external methods outside of products. The problem with the new code is that it imports all products to figure out some of their details. So for every factory dispatcher access we now have an import for each product, which means lots of filesystem accesses and python interpreter work being done. > >They came up with http://paste.plone.org/13211 which is an imho somewhat > >evil approach which introduces a thread-local cache for > >App.FactoryDispatcher._product_packages. As described below the speedup > >as a result of that is huge. > > I would not be opposed against such a speedup patch as long as it does > not raise other problems. However I don't know App.* enough to be evaluate > the patch. http://paste.plone.org/13217 should do the trick. It makes _product_packages cache its result using the list of products in Control_Panel as a cache key. That makes sure that removing or adding products there will not result in stale data being returned by the dispatcher. Wichert. -- Wichert Akkerman <[EMAIL PROTECTED]>It is simple to make things. http://www.wiggy.net/ It is hard to make things simple. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] TypesTool speedup
--On 4. März 2007 13:57:23 +0100 Wichert Akkerman <[EMAIL PROTECTED]> wrote: I'm forwarding a message from limi here, since it makes much more sense here than it does on plone-developers. To summarize it: while analying performance of the Plone 3 codebase they noticed a lot of time was spent trying to figure out which types could be added at a location. Part of that logic uses the TypesTool _queryFactoryMethod function, which goes through the dispatcher to get to the factory method for a type, which is apparently very slow in Zope 2.10, and much faster in Zope 2.9. Any idea why the code is different between 2.9 and 2.10? They came up with http://paste.plone.org/13211 which is an imho somewhat evil approach which introduces a thread-local cache for App.FactoryDispatcher._product_packages. As described below the speedup as a result of that is huge. I would not be opposed against such a speedup patch as long as it does not raise other problems. However I don't know App.* enough to be evaluate the patch. Andreas pgp4nEbEdRjxB.pgp Description: PGP signature ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] TypesTool speedup
I'm forwarding a message from limi here, since it makes much more sense here than it does on plone-developers. To summarize it: while analying performance of the Plone 3 codebase they noticed a lot of time was spent trying to figure out which types could be added at a location. Part of that logic uses the TypesTool _queryFactoryMethod function, which goes through the dispatcher to get to the factory method for a type, which is apparently very slow in Zope 2.10, and much faster in Zope 2.9. They came up with http://paste.plone.org/13211 which is an imho somewhat evil approach which introduces a thread-local cache for App.FactoryDispatcher._product_packages. As described below the speedup as a result of that is huge. Since Zope startup is as far as I know the only time the result of _product_packages is change perhaps it would be useful to generate that result on speedup. Or perhaps to make _product_packages cache its output. I certainly do want any monkey patches in Plone for this since it should be fixed in either Zope or CMF. Wichert. - Forwarded message from Alexander Limi <[EMAIL PROTECTED]> - From: Alexander Limi <[EMAIL PROTECTED]> To: [EMAIL PROTECTED] Subject: [Plone-developers] Plone 3 - now 3x faster for logged-in users Date: Sat, 03 Mar 2007 19:37:54 -0800 Message-ID: <[EMAIL PROTECTED]> Hello from the optimization sprint :) Just thought I'd keep you updated on some of the stuff we've been up to here. The main news for the day is that we found a combination of slowness in Zope 2.10 and CMF using that method in a less-than-optimal way. The result was that rendering the content menu (the one that decides what types you can add where etc) was eating an amazing amount of time. Explained in my usual UI designer hand-wavy way, every time we load a page in logged in mode, it checks the list of Products to figure out what you can add. So, by caching this locally in TypesTool so it doesn't get looked up every time (the list of Products doesn't change with every request, after all), we get a significant speedup. Let's see some numbers of loading the front page of a fresh plone : Before types tool optimization (10 requests): Products/CMFPlone/skins/plone_content/document_view.pt 8.54s lib/python/plone/app/contentmenu/contentmenu.pt6.26s After types tool optimization: Products/CMFPlone/skins/plone_content/document_view.pt 2.55s lib/python/plone/app/contentmenu/contentmenu.pt0.29s That makes every logged in page over 3 times faster. I like it. :) Patch for CMFCore TypesTool included. -- Alexander Limi · http://limi.net ___ Plone-developers mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/plone-developers - End forwarded message - -- Wichert Akkerman <[EMAIL PROTECTED]>It is simple to make things. http://www.wiggy.net/ It is hard to make things simple. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )