[Zope-dev] zope-tests - OK: 21
This is the summary for test reports received on the zope-tests list between 2012-11-05 00:00:00 UTC and 2012-11-06 00:00:00 UTC: See the footnotes for test reports of unsuccessful builds. An up-to date view of the builders is also available in our buildbot documentation: http://docs.zope.org/zopetoolkit/process/buildbots.html#the-nightly-builds Reports received Successful - zopetoolkit_trunk - Build # 68 Successful - zopetoolkit_trunk_app - Build # 52 Zope-2.10 Python-2.4.6 : Linux Zope-2.11 Python-2.4.6 : Linux Zope-2.12 Python-2.6.8 : Linux Zope-2.13 Python-2.6.8 : Linux Zope-2.13 Python-2.7.3 : Linux Zope-trunk Python-2.6.8 : Linux Zope-trunk Python-2.7.3 : Linux winbot / ZODB_dev py_265_win32 winbot / ZODB_dev py_265_win64 winbot / ZODB_dev py_270_win32 winbot / ZODB_dev py_270_win64 winbot / ztk_10 py_254_win32 winbot / ztk_10 py_265_win32 winbot / ztk_10 py_265_win64 winbot / ztk_11 py_254_win32 winbot / ztk_11 py_265_win32 winbot / ztk_11 py_265_win64 winbot / ztk_11 py_270_win32 winbot / ztk_11 py_270_win64 Non-OK results -- ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] AccessControl C extension, ExtensionBase, Cylic memory and tp_traverse
Hi Silvain, Thanks for the deep analysis. Did you put this code in a branch somewhere for us to take a look? Cheers, Leo On Tue, Nov 6, 2012 at 2:54 PM, Sylvain Viollon wrote: > Hello, > > I recently find out I had a memory leak in Silva, with Python 2.7. After > playing a while with various tools (objgraph is nice), I found out I leaked > between two and three objects at every request: > >- an AccessControl.SecurityManagement.SecurityContext instance, > >- two bound methods to ZopeSecurityPolicy.validate and > ZopeSecurityPolicy.checkPermission. > > Of course, those leaks disappears as soon as I use the Python > implementation for the security instead of the one in C. And I have tested it > as well with various older versions of AccessControl, and have the same > behavior. > > Looking closely at the code, I see that those objects are used internally > by the SecurityManager. The SecurityManager is implemented in C, using the > facilities provided by ExtensionClass. The SecurityManager properly implement > the method dealloc, and it is correctly called. However, since it holds > objects managed by the Python garbage collector (namely one called context > that is an instance of AccessControl.SecurityManagement.SecurityContext, one > called policy, that is an instance of ZopeSecurityPolicy, and a two last ones > that are the bound methods ZopeSecurityPolicy.validate and > ZopeSecurityPolicy.checkPermission), it should implements the tp_traverse > method, but it doesn't. This is not done because ExtensionClass uses the slot > for tp_traverse to store something else (namely what goes in tp_methods > later). > > In a local checkout of AccessControl, I converted the SecurityManager C > code not to rely on ExtensionClass anymore, but use the 'new object' C API, > that is mostly equivalent to ExtensionClass, and implemented the tp_traverse > method. This fixed my memory leak. > >I propose you to commit my code in a branch, and after review to merge it > with the branch 2.13 and the trunk of AccessControl. > >I don't see any argument against not using ExtensionClass anymore for > SecurityManager, and it might be possible to convert some other classes to > the 'new object' C API too. I can have a look at it. > >Regards, > >Sylvain, > > -- > Sylvain Viollon -- Infrae > t +31 10 243 7051 -- http://infrae.com > Hoevestraat 10 3033GC Rotterdam -- The Netherlands > > > > ___ > Zope-Dev maillist - Zope-Dev@zope.org > https://mail.zope.org/mailman/listinfo/zope-dev > ** No cross posts or HTML encoding! ** > (Related lists - > https://mail.zope.org/mailman/listinfo/zope-announce > https://mail.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] AccessControl C extension, ExtensionBase, Cylic memory and tp_traverse
Hello, I recently find out I had a memory leak in Silva, with Python 2.7. After playing a while with various tools (objgraph is nice), I found out I leaked between two and three objects at every request: - an AccessControl.SecurityManagement.SecurityContext instance, - two bound methods to ZopeSecurityPolicy.validate and ZopeSecurityPolicy.checkPermission. Of course, those leaks disappears as soon as I use the Python implementation for the security instead of the one in C. And I have tested it as well with various older versions of AccessControl, and have the same behavior. Looking closely at the code, I see that those objects are used internally by the SecurityManager. The SecurityManager is implemented in C, using the facilities provided by ExtensionClass. The SecurityManager properly implement the method dealloc, and it is correctly called. However, since it holds objects managed by the Python garbage collector (namely one called context that is an instance of AccessControl.SecurityManagement.SecurityContext, one called policy, that is an instance of ZopeSecurityPolicy, and a two last ones that are the bound methods ZopeSecurityPolicy.validate and ZopeSecurityPolicy.checkPermission), it should implements the tp_traverse method, but it doesn't. This is not done because ExtensionClass uses the slot for tp_traverse to store something else (namely what goes in tp_methods later). In a local checkout of AccessControl, I converted the SecurityManager C code not to rely on ExtensionClass anymore, but use the 'new object' C API, that is mostly equivalent to ExtensionClass, and implemented the tp_traverse method. This fixed my memory leak. I propose you to commit my code in a branch, and after review to merge it with the branch 2.13 and the trunk of AccessControl. I don't see any argument against not using ExtensionClass anymore for SecurityManager, and it might be possible to convert some other classes to the 'new object' C API too. I can have a look at it. Regards, Sylvain, -- Sylvain Viollon -- Infrae t +31 10 243 7051 -- http://infrae.com Hoevestraat 10 3033GC Rotterdam -- The Netherlands ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
