Re: Lazy-evaluation lists/dictionaries
On 2014-10-26, Terry Reedy tjre...@udel.edu wrote: On 10/26/2014 10:14 AM, Jon Ribbens wrote: Is there any better way to do this other than simply re-implementing these types from scratch, emulating all their methods and operations? (i.e. using UserList/UserDict). I was under the impression that that sort of thing was supposed to have gone since Python 2.2 or so. We considered dropping UserDict and UserList for 3.0 but kept them in collections for cases in which subclassing does not work. It seems on further investigation to be hard/impossible to subclass Python classes from C extensions. I've gone with subclassing dict, and reimplementing most of its methods. It helps that I only need it to be read-only. It's a pity there's no protocol for 'dynamic' lists/dicts though. -- https://mail.python.org/mailman/listinfo/python-list
Re: Lazy-evaluation lists/dictionaries
On 2014-10-26, Tim Delaney timothy.c.dela...@gmail.com wrote: On 27 October 2014 01:14, Jon Ribbens jon+use...@unequivocal.co.uk wrote: I have a need, in a Python C extension I am writing, for lists and dictionaries with lazy evaluation - by which I mean that at least some of the values in the lists/dictionaries are proxy objects which, rather than returning as themselves, should return the thing they are a proxy for when retrieved. This is because retrieving the proxied objects is expensive and only a small minority of them will actually be accessed, so retrieving them all before they are actually accessed is massively inefficient. Why not put proxy objects into the list/dict? That's precisely what I am doing. The point is that when they are retrieved they need to be resolved into the genuine objects. Have a look at the weakref module for an API that may be suitable for such proxy objects (if you used the same API, that would also allow you to transparently use weakrefs in your lists/dicts). Hmm, the idea behind that appears to be to create a proxy that emulates every possible method of every conceivable type. My method of only emulating the list and dict methods seems to be somewhat simpler for my purpose ;-) -- https://mail.python.org/mailman/listinfo/python-list
Lazy-evaluation lists/dictionaries
I have a need, in a Python C extension I am writing, for lists and dictionaries with lazy evaluation - by which I mean that at least some of the values in the lists/dictionaries are proxy objects which, rather than returning as themselves, should return the thing they are a proxy for when retrieved. This is because retrieving the proxied objects is expensive and only a small minority of them will actually be accessed, so retrieving them all before they are actually accessed is massively inefficient. With object attributes, this can be easily done with the descriptor protocol and having properties which look like simple objects but actually cause a method to be invoked when they are accessed. However there doesn't seem to be any equivalent way of doing this for retrieving items from lists or dictionaries - unfortunately, the method implementations of list() and dict() are full of direct accesses to the underlying data pointers rather than indirecting through obj-tp_as_mapping-mp_subscript or whatever. Is there any better way to do this other than simply re-implementing these types from scratch, emulating all their methods and operations? (i.e. using UserList/UserDict). I was under the impression that that sort of thing was supposed to have gone since Python 2.2 or so. -- https://mail.python.org/mailman/listinfo/python-list
Re: Lazy-evaluation lists/dictionaries
On 27 October 2014 01:14, Jon Ribbens jon+use...@unequivocal.co.uk wrote: I have a need, in a Python C extension I am writing, for lists and dictionaries with lazy evaluation - by which I mean that at least some of the values in the lists/dictionaries are proxy objects which, rather than returning as themselves, should return the thing they are a proxy for when retrieved. This is because retrieving the proxied objects is expensive and only a small minority of them will actually be accessed, so retrieving them all before they are actually accessed is massively inefficient. Why not put proxy objects into the list/dict? Have a look at the weakref module for an API that may be suitable for such proxy objects (if you used the same API, that would also allow you to transparently use weakrefs in your lists/dicts). Tim Delaney -- https://mail.python.org/mailman/listinfo/python-list
Re: Lazy-evaluation lists/dictionaries
On 10/26/2014 10:14 AM, Jon Ribbens wrote: I have a need, in a Python C extension I am writing, for lists and dictionaries with lazy evaluation - by which I mean that at least some of the values in the lists/dictionaries are proxy objects which, rather than returning as themselves, should return the thing they are a proxy for when retrieved. This is because retrieving the proxied objects is expensive and only a small minority of them will actually be accessed, so retrieving them all before they are actually accessed is massively inefficient. With object attributes, this can be easily done with the descriptor protocol and having properties which look like simple objects but actually cause a method to be invoked when they are accessed. However there doesn't seem to be any equivalent way of doing this for retrieving items from lists or dictionaries - unfortunately, the method implementations of list() and dict() are full of direct accesses to the underlying data pointers rather than indirecting through obj-tp_as_mapping-mp_subscript or whatever. Is there any better way to do this other than simply re-implementing these types from scratch, emulating all their methods and operations? (i.e. using UserList/UserDict). I was under the impression that that sort of thing was supposed to have gone since Python 2.2 or so. We considered dropping UserDict and UserList for 3.0 but kept them in collections for cases in which subclassing does not work. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
