[Zope] BaseRequest.traverse, skins and BrowserViews
The current setup of traversal seems to give precedence to objects in a skin, over a browserview... but only at the siteroot. Is this by design or accident? If by accident and not knowing the innards that well, is it possible to have the opposite occur? As in, I want to override search.pt from default Plone skins with my own browserview version. Seemed a reasonable thing to want to do to me anyway. Dylan. - Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://sourceforge.net/services/buy/index.php ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Re: UnpickleableError: Cannot pickleobjects
"Dylan Jay" <[EMAIL PROTECTED]> wrote in message
news:[EMAIL PROTECTED]
>I think I've tracked it down to a RamCacheManager inside a volitile
>variable. But I thought volitile variables we'ren't stored? Am I doing
>something strange with my heirachy that is abusing the ZODB? The heirachy
>is
> object.[Cacheable] And the Cacheable has _v_ZCacheable_cache =
> RamCacheManager. I believe the RamCacheManager has locks in it.
I think I've tracked down the problem. Cacheable did not inherit from
Persistent. I was infact using a super class of Cacheable called
NamedCacheable which inherited Acquisition.Implicit which I'd thought was
enough (a long time ago when I wrote it).
Just for the record, how I tracked this down was by using dumps. In pdb I
did
from pickle import dumps
pp state['attributename']
until I found the parts of the state that raised errors.
>
>
>> "Dieter Maurer" <[EMAIL PROTECTED]> wrote in message
>> news:[EMAIL PROTECTED]
>>> Dylan Jay wrote at 2005-6-14 17:45 +1000:
>>>> ...
>>>>356, in _commit_objects
>>>>jar.commit(o, self)
>>>> File "C:\Program
>>>> Files\Zope-2.7.5-final\lib\python\ZODB\Connection.py",
>>>>line 4
>>>>52, in commit
>>>>dump(state)
>>>>UnpickleableError: Cannot pickle objects
>>>
>>> I would catch the error and enter a debugger to analyse the "o" above.
>>>
>>> With "pdb" (the Python Debugger, part of Python),
>>> this could look like:
>>>
>>> try: dump(state)
>>> except: import pdb; pdb.set_trace()
>>
>> Good suggestion (I did try to get debugging working in WingIDE and Boa
>> but your way worked best in the end).
>> Unfortunatly I'm still none the wiser.
>> I know which object it is and I've looked at its state (below). I've
>> looked at the OOBTree and made sure it contains nothing that has a lock
>> in it. I can look at sys.exc_info[1].args which is a lock object but what
>> I really need to do is work out where this lock object is and why its
>> entangled with my transaction. Any suggestions?
>>
>> {'__ac_local_roles__': {'djay': ['Owner']},
>> '_cacheables': {'constructReviewer': > 0409A8D0>,
>> 'findNeighbors': },
>> '_hasBeenTrained': 0,
>> 'filterByMovies': 0,
>> 'filterByPeople': 0,
>> 'filterByVotes': 0,
>> 'hashMoviesToUse': None,
>> 'hashPeopleToUse': None,
>> 'hoods': ,
>> 'id': 'predictor',
>> 'keepInMemory': 0,
>> 'moviesToUse': None,
>> 'neighborhoodSize': 30,
>> 'overlapPenalty': 50,
>> 'peopleToUse': None,
>> 'reviewers': None,
>> 'title': '',
>> 'votesToUse': None}
>>
>>
>>
>> ___
>> Zope maillist - Zope@zope.org
>> http://mail.zope.org/mailman/listinfo/zope
>> ** No cross posts or HTML encoding! **
>> (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce
>> http://mail.zope.org/mailman/listinfo/zope-dev )
>>
>
>
>
> ___
> Zope maillist - Zope@zope.org
> http://mail.zope.org/mailman/listinfo/zope
> ** No cross posts or HTML encoding! **
> (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce
> http://mail.zope.org/mailman/listinfo/zope-dev )
>
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Re: UnpickleableError: Cannot pickle objects
I think I've tracked it down to a RamCacheManager inside a volitile
variable. But I thought volitile variables we'ren't stored? Am I doing
something strange with my heirachy that is abusing the ZODB? The heirachy is
object.[Cacheable] And the Cacheable has _v_ZCacheable_cache =
RamCacheManager. I believe the RamCacheManager has locks in it.
> "Dieter Maurer" <[EMAIL PROTECTED]> wrote in message
> news:[EMAIL PROTECTED]
>> Dylan Jay wrote at 2005-6-14 17:45 +1000:
>>> ...
>>>356, in _commit_objects
>>>jar.commit(o, self)
>>> File "C:\Program Files\Zope-2.7.5-final\lib\python\ZODB\Connection.py",
>>>line 4
>>>52, in commit
>>>dump(state)
>>>UnpickleableError: Cannot pickle objects
>>
>> I would catch the error and enter a debugger to analyse the "o" above.
>>
>> With "pdb" (the Python Debugger, part of Python),
>> this could look like:
>>
>> try: dump(state)
>> except: import pdb; pdb.set_trace()
>
> Good suggestion (I did try to get debugging working in WingIDE and Boa but
> your way worked best in the end).
> Unfortunatly I'm still none the wiser.
> I know which object it is and I've looked at its state (below). I've
> looked at the OOBTree and made sure it contains nothing that has a lock in
> it. I can look at sys.exc_info[1].args which is a lock object but what I
> really need to do is work out where this lock object is and why its
> entangled with my transaction. Any suggestions?
>
> {'__ac_local_roles__': {'djay': ['Owner']},
> '_cacheables': {'constructReviewer': 0409A8D0>,
> 'findNeighbors': },
> '_hasBeenTrained': 0,
> 'filterByMovies': 0,
> 'filterByPeople': 0,
> 'filterByVotes': 0,
> 'hashMoviesToUse': None,
> 'hashPeopleToUse': None,
> 'hoods': ,
> 'id': 'predictor',
> 'keepInMemory': 0,
> 'moviesToUse': None,
> 'neighborhoodSize': 30,
> 'overlapPenalty': 50,
> 'peopleToUse': None,
> 'reviewers': None,
> 'title': '',
> 'votesToUse': None}
>
>
>
> ___
> Zope maillist - Zope@zope.org
> http://mail.zope.org/mailman/listinfo/zope
> ** No cross posts or HTML encoding! **
> (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce
> http://mail.zope.org/mailman/listinfo/zope-dev )
>
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Re: UnpickleableError: Cannot pickle objects
"Dieter Maurer" <[EMAIL PROTECTED]> wrote in message
news:[EMAIL PROTECTED]
> Dylan Jay wrote at 2005-6-14 17:45 +1000:
>> ...
>>356, in _commit_objects
>>jar.commit(o, self)
>> File "C:\Program Files\Zope-2.7.5-final\lib\python\ZODB\Connection.py",
>>line 4
>>52, in commit
>>dump(state)
>>UnpickleableError: Cannot pickle objects
>
> I would catch the error and enter a debugger to analyse the "o" above.
>
> With "pdb" (the Python Debugger, part of Python),
> this could look like:
>
> try: dump(state)
> except: import pdb; pdb.set_trace()
Good suggestion (I did try to get debugging working in WingIDE and Boa but
your way worked best in the end).
Unfortunatly I'm still none the wiser.
I know which object it is and I've looked at its state (below). I've looked
at the OOBTree and made sure it contains nothing that has a lock in it. I
can look at sys.exc_info[1].args which is a lock object but what I really
need to do is work out where this lock object is and why its entangled with
my transaction. Any suggestions?
{'__ac_local_roles__': {'djay': ['Owner']},
'_cacheables': {'constructReviewer': ,
'findNeighbors': },
'_hasBeenTrained': 0,
'filterByMovies': 0,
'filterByPeople': 0,
'filterByVotes': 0,
'hashMoviesToUse': None,
'hashPeopleToUse': None,
'hoods': ,
'id': 'predictor',
'keepInMemory': 0,
'moviesToUse': None,
'neighborhoodSize': 30,
'overlapPenalty': 50,
'peopleToUse': None,
'reviewers': None,
'title': '',
'votesToUse': None}
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] UnpickleableError: Cannot pickle objects
I'm having some ZODB troubles. I'm getting the following error. The problem is I'm not using locks in the objects that are being changed. I do have a lock being used with the Schedule object which is in the acquisition path. but this lock was a module variable rather than a attribute so that shouldn't matter should it? I've also taken the lock code out altogeather now and I'm still getting the same error. Any ideas? 2005-06-14T17:33:39 ERROR(200) SiteError http://localhost:8080/iv/scheduler/noti fyOne Traceback (most recent call last): File "C:\Program Files\Zope-2.7.5-final\lib\python\ZPublisher\Publish.py", lin e 107, in publish transactions_manager.commit() File "C:\Program Files\Zope-2.7.5-final\lib\python\Zope\App\startup.py", line 222, in commit get_transaction().commit() File "C:\Program Files\Zope-2.7.5-final\lib\python\ZODB\Transaction.py", line 241, in commit ncommitted += self._commit_objects(objects) File "C:\Program Files\Zope-2.7.5-final\lib\python\ZODB\Transaction.py", line 356, in _commit_objects jar.commit(o, self) File "C:\Program Files\Zope-2.7.5-final\lib\python\ZODB\Connection.py", line 4 52, in commit dump(state) UnpickleableError: Cannot pickle objects -- Dylan Jay --- http://www.meetmemap.com - makes giving directions easy http://www.pretaweb.com - Update-it-yourself, low cost websites http://www.eatmanifesto.com - against all dumbing down of food Y!:dylan_jay ICQ:520341 MSN:[EMAIL PROTECTED] Ph:+61423198306 ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] zope vs. memory [was Re: Refcounts, ZODB Cache and memory leaks?]
Just in case anyone can shed anymore light on zope memory woes, here is where I've got to in my investigation: - Memory grows and mostly doesn't shrink. - Its not due to objects that can't be collected by the GC - Its not due to the ZODB cache That leaves the following causes I think - RAMCache etc and module/class variables. Either mine, plone's or zope's. - Pythons design of not releasing memory from small objects back to the OS http://www.sauria.com/~twl/conferences/pycon2005/20050325/Improving%20Python's%20Memory%20Allocator.html Has anyone tried to determine how much of an average catalog search is <256kb? I'm guessing a lot. - Some other kind of reference being help somewhere in the zope machinery. Any tips anyone? >> "Peter Bengtsson" <[EMAIL PROTECTED]> wrote in message >> news:[EMAIL PROTECTED] >>> Dylan Jay dylanjay.com> writes: >>> >>>> If anyone has experience with finding these kind of rouge references >>>> I'd >>>> really appreciate any tips. >>> >>> I have no idea how to patch tings with LeakFinder to find your problems >>> because >>> LeakFinder, albeit a great piece of code, attempts an incredibly >>> difficult task. >>> Most likely, your memory leaks is due to an assignment of really large >>> lists on >>> complex objects (eg. more complex than strings or numbers). >> >> Can you expound on this a little as this sounds very interesting? Why do >> assignments to a list normally result in leaks? Do you mean lists of >> complex objects or complex objects that have lists? >> >>> Look for any constructs where you extract large lists and do operations >>> on them. >>> Once you have, if you ever, contain these, return to the mailing list >>> and we can >>> try to solve it together. >> >> I certainly do operations on lists, partucularly LazyMap etc. I do >> operations with catalog results and map them to lazily to custom objects >> which are joined by aquisition to catalog brains. >> >> I'm new to memory leak debugging in python but have done it before in >> java. >>>From what I understand garbage collection happens differently in java. In >> java you have to avoid circular references, is the same true in python? >> >> >>> >>> ___ >>> Zope maillist - Zope@zope.org >>> http://mail.zope.org/mailman/listinfo/zope >>> ** No cross posts or HTML encoding! ** >>> (Related lists - >>> http://mail.zope.org/mailman/listinfo/zope-announce >>> http://mail.zope.org/mailman/listinfo/zope-dev ) >>> >> >> >> >> ___ >> Zope maillist - Zope@zope.org >> http://mail.zope.org/mailman/listinfo/zope >> ** No cross posts or HTML encoding! ** >> (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce >> http://mail.zope.org/mailman/listinfo/zope-dev ) >> > > > > ___ > Zope maillist - Zope@zope.org > http://mail.zope.org/mailman/listinfo/zope > ** No cross posts or HTML encoding! ** > (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce > http://mail.zope.org/mailman/listinfo/zope-dev ) > ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Re: Refcounts, ZODB Cache and memory leaks?
I've been going some testing with gc.set_debug etc and it looks like there isn't a memory leak or at least not one the garbage collector can find. That means that the data must be remaining reachable somehow. -- Dylan Jay --- http://www.meetmemap.com - makes giving directions easy http://www.pretaweb.com - Update-it-yourself, low cost websites http://www.eatmanifesto.com - against all dumbing down of food Y!:dylan_jay ICQ:520341 MSN:[EMAIL PROTECTED] Ph:+61423198306 "Dylan Jay" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > "Peter Bengtsson" <[EMAIL PROTECTED]> wrote in message > news:[EMAIL PROTECTED] >> Dylan Jay dylanjay.com> writes: >> >>> If anyone has experience with finding these kind of rouge references I'd >>> really appreciate any tips. >> >> I have no idea how to patch tings with LeakFinder to find your problems >> because >> LeakFinder, albeit a great piece of code, attempts an incredibly >> difficult task. >> Most likely, your memory leaks is due to an assignment of really large >> lists on >> complex objects (eg. more complex than strings or numbers). > > Can you expound on this a little as this sounds very interesting? Why do > assignments to a list normally result in leaks? Do you mean lists of > complex objects or complex objects that have lists? > >> Look for any constructs where you extract large lists and do operations >> on them. >> Once you have, if you ever, contain these, return to the mailing list and >> we can >> try to solve it together. > > I certainly do operations on lists, partucularly LazyMap etc. I do > operations with catalog results and map them to lazily to custom objects > which are joined by aquisition to catalog brains. > > I'm new to memory leak debugging in python but have done it before in > java. >>From what I understand garbage collection happens differently in java. In > java you have to avoid circular references, is the same true in python? > > >> >> ___ >> Zope maillist - Zope@zope.org >> http://mail.zope.org/mailman/listinfo/zope >> ** No cross posts or HTML encoding! ** >> (Related lists - >> http://mail.zope.org/mailman/listinfo/zope-announce >> http://mail.zope.org/mailman/listinfo/zope-dev ) >> > > > > ___ > Zope maillist - Zope@zope.org > http://mail.zope.org/mailman/listinfo/zope > ** No cross posts or HTML encoding! ** > (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce > http://mail.zope.org/mailman/listinfo/zope-dev ) > ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Re: Refcounts, ZODB Cache and memory leaks?
"Peter Bengtsson" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > Dylan Jay dylanjay.com> writes: > >> If anyone has experience with finding these kind of rouge references I'd >> really appreciate any tips. > > I have no idea how to patch tings with LeakFinder to find your problems > because > LeakFinder, albeit a great piece of code, attempts an incredibly difficult > task. > Most likely, your memory leaks is due to an assignment of really large > lists on > complex objects (eg. more complex than strings or numbers). Can you expound on this a little as this sounds very interesting? Why do assignments to a list normally result in leaks? Do you mean lists of complex objects or complex objects that have lists? > Look for any constructs where you extract large lists and do operations on > them. > Once you have, if you ever, contain these, return to the mailing list and > we can > try to solve it together. I certainly do operations on lists, partucularly LazyMap etc. I do operations with catalog results and map them to lazily to custom objects which are joined by aquisition to catalog brains. I'm new to memory leak debugging in python but have done it before in java. >From what I understand garbage collection happens differently in java. In java you have to avoid circular references, is the same true in python? > > ___ > Zope maillist - Zope@zope.org > http://mail.zope.org/mailman/listinfo/zope > ** No cross posts or HTML encoding! ** > (Related lists - > http://mail.zope.org/mailman/listinfo/zope-announce > http://mail.zope.org/mailman/listinfo/zope-dev ) > ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Refcounts, ZODB Cache and memory leaks?
Hi, I'm trying to track down some memory leaks using LeakFinder. I get some custom objects with refcounts that keep going up on use. When I patch them using leakfinder I get no results which should be correct as they should be loaded from the ZODB rather than created. The problem is, why won't they go away after GC or clearing the ZODB cache? If anyone has experience with finding these kind of rouge references I'd really appreciate any tips. Dylan Jay Web Builder [EMAIL PROTECTED] --- www.pretaweb.com - Reliable & responsive CMS website development www.meetmemap.com - makes giving directions easy www.eatmanifesto.com - Traditional Sustainable and Delectable Ph:+61(0)423198306 ~ Y!+Skype:dylan_jay ~ ICQ:520341 ~ SIP:[EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Tips in memory leaks esp with Catalogs?
Hi Everyone, I have an application that gains a large memory footprint very quickly, in casual testing 100M in an hour or so. I have a largeish ZCatalog with extra PathIndexes, using Plone, Archetypes and various other custom code. Using LeakFinder I get lots of BTree stuff from ZCatalog queries. It looks like it's being created legitimately but what's keeping the references to them and how to go about finding out? In the java world I would use a memory profiler to follow the reference chain till it went where it shouldn't. Is there such a thing for python? If not how does one go about working out whats keeping the memory? BTrees.IOBTree.IOBucket 18479 4476 BTrees.OOBTree.OOBucket 95871992 BTrees.IIBTree.IISet52731172 DateTime.DateTime.DateTime 3725424 Acquisition.ImplicitAcquirerWrapper 1231218 Products.PlacelessTranslationService.GettextMessageCatalog.GettextMessageCat alog641 212 AccessControl.User.User 276 196 Dylan Jay Web Builder [EMAIL PROTECTED] --- www.pretaweb.com - Reliable & responsive CMS website development www.meetmemap.com - makes giving directions easy www.eatmanifesto.com - Traditional Sustainable and Delectable Ph:+61(0)423198306 ~ Y!+Skype:dylan_jay ~ ICQ:520341 ~ SIP:[EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
