Re: Question about weakref
On 05/07/2012 19:47, Dieter Maurer wrote: Frank Millmanfr...@chagford.com writes: I would still like to know why weakref.proxy raised an exception. I have re-read the manual several times, and googled for similar problems, but am none the wiser. In fact, it is documented. Accessing a proxy will raise an exception when the proxied object no longer exists. What you can ask is why your proxy has been accessed after the object was deleted. The documentation is specific: during the callback, the object should still exist. Thus, apparently, one of your proxies outlived an event that should have deleted it (probably a hole in your logic). I have investigated a bit further, and now I have a clue as to what is happening, though not a full understanding. If you use 'b = weakref.ref(obj)', 'b' refers to the weak reference, and 'b()' refers to the referenced object. If you use 'b = weakref.proxy(obj)', 'b' refers to the referenced object. I don't know how to refer to the weak reference itself. In a way that is the whole point of using 'proxy', but the difficulty comes when you want to remove the weak reference when the referenced object is deleted. This is from the manual section on weakref.ref - If callback is provided and not None, and the returned weakref object is still alive, the callback will be called when the object is about to be finalized; the weak reference object will be passed as the only parameter to the callback; the referent will no longer be available. My callback method looks like this - del del_b(b): self.array.remove(b) It successfully removes the weak reference from self.array. This is from the manual section on weakref.proxy - callback is the same as the parameter of the same name to the ref() function. My callback method looks the same. However, although 'b' is the weak reference, when I refer to 'b' it refers to the original object, which at this stage no longer exists. So my revised question is - How can you remove the weak reference if you use proxy? The full story is more complicated than that - why does my example work when I delete x, then y, then z, but not if I reverse the order? However, I think that I have isolated the fundamental reason. So any comments on my latest findings will be appreciated. Frank -- http://mail.python.org/mailman/listinfo/python-list
Re: Question about weakref
On Fri, Jul 6, 2012 at 1:00 AM, Frank Millman fr...@chagford.com wrote: I have investigated a bit further, and now I have a clue as to what is happening, though not a full understanding. If you use 'b = weakref.ref(obj)', 'b' refers to the weak reference, and 'b()' refers to the referenced object. If you use 'b = weakref.proxy(obj)', 'b' refers to the referenced object. I don't know how to refer to the weak reference itself. In a way that is the whole point of using 'proxy', but the difficulty comes when you want to remove the weak reference when the referenced object is deleted. Not quite. 'b' refers to the proxy, which uses magic methods to mimic the referenced object. It is still a separate object, however. In fact, I actually think it's not directly possible to refer to the *referenced object* via a proxy, although there are round-about ways to accomplish it. import weakref class Foo(object): pass a = Foo() id(a) 11253072 b = weakref.proxy(a) id(b) 11258400 a is a True a is b False The full story is more complicated than that - why does my example work when I delete x, then y, then z, but not if I reverse the order? On that, I'm really not sure. I tried to reproduce the problem locally and wasn't able to. What build of Python are you using, and on what platform? I have one suggestion, though: you might try removing the __del__ method from the listener class, as the presence of that method can interfere with garbage collection in some cases, and it is generally contra-recommended. I'm not sure why that would affect the code you posted, but it can't hurt to try it. Cheers, Ian -- http://mail.python.org/mailman/listinfo/python-list
Re: Question about weakref
On Fri, Jul 6, 2012 at 11:04 AM, Ian Kelly ian.g.ke...@gmail.com wrote: On that, I'm really not sure. I tried to reproduce the problem locally and wasn't able to. What build of Python are you using, and on what platform? I spoke too soon, I am able to reproduce it. I think what's going on here is that when you try to remove the proxy from the list, the list.remove() call searches for the object by *equality*, not by identity. The problem is that at the time of the callback, the referent is no longer available to implement the equality test, as noted in the weakref.ref() documentation. As long as the proxy happens to be the first element of the list, this is not a problem, because the proxy evidently short-circuits self == self to return True. If it's not the first element of the list, though, then the first comparison compares the proxy to some other object, and the proxy raises an exception, because without the referent it no longer knows how to compare. If you change your del_b() method to the following, though, it works: def del_b(self, b): for i, x in enumerate(self.array): if b is x: del self.array[i] This works because it carefully only handles the proxy object itself and no longer relies on any aspect of the referent for deletion. It's not a problem for weakref.ref, because ref objects require an explicit dereferencing step to access the referent. Cheers, Ian -- http://mail.python.org/mailman/listinfo/python-list
Re: Question about weakref
On Fri, Jul 6, 2012 at 11:48 AM, Ian Kelly ian.g.ke...@gmail.com wrote: def del_b(self, b): for i, x in enumerate(self.array): if b is x: del self.array[i] That should probably have an explicit break on the end: def del_b(self, b): for i, x in enumerate(self.array): if b is x: del self.array[i] break -- http://mail.python.org/mailman/listinfo/python-list
Re: Question about weakref
Ian Kelly wrote: def del_b(self, b): for i, x in enumerate(self.array): if b is x: del self.array[i] break Nice work, Ian. -- http://mail.python.org/mailman/listinfo/python-list
Re: Question about weakref
On 06/07/2012 20:12, Ethan Furman wrote: Ian Kelly wrote: def del_b(self, b): for i, x in enumerate(self.array): if b is x: del self.array[i] break Nice work, Ian. I second that. Thanks very much, Ian. Frank -- http://mail.python.org/mailman/listinfo/python-list
Re: Question about weakref
Frank Millman fr...@chagford.com writes: I have a situation where I thought using weakrefs would save me a bit of effort. Instead of the low level weakref, you might use a WeakKeyDictionary. -- http://mail.python.org/mailman/listinfo/python-list
Re: Question about weakref
On 05/07/2012 10:46, Dieter Maurer wrote: Frank Millmanfr...@chagford.com writes: I have a situation where I thought using weakrefs would save me a bit of effort. Instead of the low level weakref, you might use a WeakKeyDictionary. Thanks, Dieter. I could do that. In fact, a WeakSet suits my purposes better. I tested it with my original example, and it works correctly. It also saves me the step of deleting the weak reference once the original object is deleted, as it seems to do that automatically. I just need to double-check that I would never have the same listener-object try to register itself with the publisher twice, as that would obviously fail with a Set, as it would with a Dict. I would still like to know why weakref.proxy raised an exception. I have re-read the manual several times, and googled for similar problems, but am none the wiser. Naturally I feel a bit uneasy using a feature of the language which sometimes fails mysteriously, so if anyone has an explanation, I would really appreciate it. Frank -- http://mail.python.org/mailman/listinfo/python-list
Re: Question about weakref
Frank Millman fr...@chagford.com writes: On 05/07/2012 10:46, Dieter Maurer wrote: Instead of the low level weakref, you might use a WeakKeyDictionary. Thanks, Dieter. I could do that. In fact, a WeakSet suits my purposes better. I tested it with my original example, and it works correctly. It also saves me the step of deleting the weak reference once the original object is deleted, as it seems to do that automatically. I just need to double-check that I would never have the same listener-object try to register itself with the publisher twice, as that would obviously fail with a Set, as it would with a Dict. No need to verify. A secondary subscription would be effectively a no-operation -- with both a set and a dict. I would still like to know why weakref.proxy raised an exception. I have re-read the manual several times, and googled for similar problems, but am none the wiser. In fact, it is documented. Accessing a proxy will raise an exception when the proxied object no longer exists. What you can ask is why your proxy has been accessed after the object was deleted. The documentation is specific: during the callback, the object should still exist. Thus, apparently, one of your proxies outlived an event that should have deleted it (probably a hole in your logic). -- http://mail.python.org/mailman/listinfo/python-list
Question about weakref
Hi all
I have a situation where I thought using weakrefs would save me a bit of
effort.
I have a basic publish/subscribe scenario. The publisher maintains a
list of listener objects, and has a method whereby a listener can
subscribe to the list by passing in 'self', whereupon it gets added to
the list.
When the publisher has something to say, it calls a pre-defined method
on each of the listeners. Simple, but it works.
The listeners are fairly transient, so when they go out of scope, I need
to remove them from the list maintained by the publisher. Instead of
keeping track of all of them and removing them explicitly, I thought of
using weakrefs and let them be removed automatically.
It almost works. Here is an example -
import weakref
class A: # the publisher class
def __init__(self):
self.array = []
def add_b(self, b):
self.array.append(weakref.ref(b, self.del_b))
def del_b(self, b):
self.array.remove(b)
def chk_b(self, ref):
for b in self.array:
b().hallo(ref)
class B: # the listener class
def __init__(self, a, name):
self.name = name
a.add_b(self)
def hallo(self, ref):
print(self.name, 'hallo from', ref)
def __del__(self):
print('%s deleted' % self.name)
a = A()
x = B(a, 'test x')
y = B(a, 'test y')
z = B(a, 'test z')
a.chk_b(1)
del x
a.chk_b(2)
del y
a.chk_b(3)
del z
a.chk_b(4)
print(a.array)
The output is as expected -
test x hallo from 1
test y hallo from 1
test z hallo from 1
test x deleted
test y hallo from 2
test z hallo from 2
test y deleted
test z hallo from 3
test z deleted
[]
Then I tried weakref.proxy.
I changed
self.array.append(weakref.ref(b, self.del_b))
to
self.array.append(weakref.proxy(b, self.del_b))
and
b().hallo(ref)
to
b.hallo(ref)
I got the same result.
Then I varied the order of deletion - instead of x, then y, then z, I
tried x, then z, then y.
Now I get the following traceback -
test x hallo from 1
test y hallo from 1
test z hallo from 1
test x deleted
test y hallo from 2
test z hallo from 2
Exception ReferenceError: 'weakly-referenced object no longer exists' in
bound
method A.del_b of __main__.A object at 0x00A8A750 ignored
test z deleted
test y hallo from 3
Traceback (most recent call last):
File F:\junk\weaklist.py, line 70, in module
a.chk_b(3)
File F:\junk\weaklist.py, line 51, in chk_b
b.hallo(ref)
ReferenceError: weakly-referenced object no longer exists
test y deleted
If I go back to using weakref.ref, but with the new deletion order, it
works.
So now I am confused.
1. Why do I get the traceback?
2. Can I rely on using weakref.ref, or does that also have some problem
that has just not appeared yet?
Any advice will be appreciated.
BTW, I am using python 3.2.2.
Thanks
Frank Millman
--
http://mail.python.org/mailman/listinfo/python-list
