[issue21145] Add the @cached_property decorator

STINNER Victor added the comment: FYI there is discussion in bpo-34995 about the usage of @cached_property with abstract methods. -- nosy: +vstinner ___ Python tracker ___

[issue21145] Add the @cached_property decorator

Carl Meyer added the comment: Thanks everyone for the thoughtful and careful reviews! Patch is much improved from where it started. And thanks Nick for merging. -- ___ Python tracker

[issue21145] Add the @cached_property decorator

Nick Coghlan added the comment: This has now been merged. Thanks for the multiple iterations on the implementation Carl, and thanks for the original proposal, Omer! -- resolution: -> fixed stage: patch review -> resolved status: open -> closed

[issue21145] Add the @cached_property decorator

Nick Coghlan added the comment: New changeset d658deac6060ee92b449a3bf424b460eafd99f3e by Nick Coghlan (Carl Meyer) in branch 'master': bpo-21145: Add cached_property decorator in functools (#6982) https://github.com/python/cpython/commit/d658deac6060ee92b449a3bf424b460eafd99f3e --

[issue21145] Add the @cached_property decorator

Change by Sergey Fedoseev : -- nosy: +sir-sigurd ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue21145] Add the @cached_property decorator

Carl Meyer added the comment: Oops, never mind; closed mine as dupe. -- ___ Python tracker ___

[issue21145] Add the @cached_property decorator

Carl Meyer added the comment: Makes sense to me. Sounds like a separate issue and PR; I filed issue33577 and will work on a patch. -- ___ Python tracker

[issue21145] Add the @cached_property decorator

Nick Coghlan added the comment: I filed https://bugs.python.org/issue33576 to cover removing the exception wrapping from __set_name__ errors. -- ___ Python tracker

[issue21145] Add the @cached_property decorator

Nick Coghlan added the comment: I think it would make sense to remove the exception wrapping from the __set_name__ calls - I don't think we're improving the ease of understanding the tracebacks by converting everything to a generic RuntimeError, and we're hurting the UX

[issue21145] Add the @cached_property decorator

Carl Meyer added the comment: Sent a PR with the patch. Nick, I tried your `__set_name__` proposal to get an earlier error in case of an object with slots, but it has the downside that Python seems to always raise a new chained exception if `__set_name__` raises any

[issue21145] Add the @cached_property decorator

Change by Carl Meyer : -- pull_requests: +6636 stage: -> patch review ___ Python tracker ___

[issue21145] Add the @cached_property decorator

Change by STINNER Victor : -- nosy: -vstinner ___ Python tracker ___ ___

[issue21145] Add the @cached_property decorator

Antoine Pitrou added the comment: I agree this would be a useful addition to the stdlib. The code might seem reasonably short, but implementing new descriptors is an advanced topic (I'd rather avoid it myself). -- ___ Python

[issue21145] Add the @cached_property decorator

Carl Meyer added the comment: > a way to invalidate or clear the cache This is already supported by the simple implementation in the patch, it's spelled `del obj.the_cached_property`. > mock patching the underlying function for testing This is easy to do with the current

[issue21145] Add the @cached_property decorator

Raymond Hettinger added the comment: FWIW, over the past decade, I've used variants of CachedProperty a number of times and have often had issues that later required messing with its internals (needing a way to invalidate or clear the cache, mock patching the

[issue21145] Add the @cached_property decorator

Omer Katz added the comment: I believe so. Let's do it :) ‫בתאריך יום ב׳, 14 במאי 2018 ב-19:30 מאת ‪Carl Meyer‬‏ <‪ rep...@bugs.python.org‬‏>:‬ > > Carl Meyer added the comment: > > > I don't think it makes sense to try to make cached_property itself

[issue21145] Add the @cached_property decorator

Carl Meyer added the comment: > I don't think it makes sense to try to make cached_property itself work > implicitly with both normal attributes and slot entries - instead, > cached_property can handle the common case as simply and efficiently as > possible, and the

[issue21145] Add the @cached_property decorator

Nick Coghlan added the comment: Having just said that I don't see much use for lazily initialized slots, it does occur to me that __weakref__ is essentially such a slot, and __dict__ itself could usefully be such a slot for types where instances mostly have a fixed set of attributes, but

[issue21145] Add the @cached_property decorator

Nick Coghlan added the comment: I realised that PEP 487's __set_name__ can be used to detect the `__slots__` conflict at class definition time rather than on first lookup: def __set_name__(self, owner, name): try: slots = owner.__slots__ except AttributeError:

[issue21145] Add the @cached_property decorator

Carl Meyer added the comment: Speaking of this hypothetical C version, what's the current policy on shipping stdlib C code that can't be emulated in pure Python? (I'm thinking of e.g. PyPy). -- ___ Python tracker

[issue21145] Add the @cached_property decorator

Carl Meyer added the comment: Thanks, Danny. Uploaded a version of the patch that adds thread-safety (with a test). Unlike in your lib, I didn't make it a separate version of the decorator; since the lock is not checked on cached access, its slight overhead on the initial computation is

[issue21145] Add the @cached_property decorator

Daniel Greenfeld added the comment: I'm delighted to see a patch submitted, but I'm concerned that it isn't thread safe. This was implemented in the cached-property package I maintain: * https://github.com/pydanny/cached-property/issues/6 * https://github.com/pydanny/cached-property/pull/9 *

[issue21145] Add the @cached_property decorator

Carl Meyer added the comment: (We wouldn't be able to match the set/delete behavior in a slots-supporting fallback implemented in Python.) -- ___ Python tracker

[issue21145] Add the @cached_property decorator

Carl Meyer added the comment: Uploaded a patch updated per Nick's comment. Not opposed to waiting to see if someone is motivated to implement a version in C that supports __slots__, but if that doesn't happen by the Python 3.7 feature deadline, I don't think it should block this proven

[issue21145] Add the @cached_property decorator

Changes by Carl Meyer : Added file: http://bugs.python.org/file45450/cached_property.diff ___ Python tracker ___

[issue21145] Add the @cached_property decorator

Serhiy Storchaka added the comment: Please no need to rush. We have 1.5 years to do this right. I believe supporting __slots__ is very important. But this is not easy task. Seems this requires C implementation. We should also design the behavior in case of setting or deleting cached

[issue21145] Add the @cached_property decorator

Carl Meyer added the comment: Makes sense, Nick, thanks. The current error message for that situation is pretty cryptic indeed. I'll update the patch with your suggestion. Do you think a documentation note about slots is also warranted? -- ___

[issue21145] Add the @cached_property decorator

Nick Coghlan added the comment: Carl's patch looks good to me, but my one request in relation to the __slots__ situation would be to give it a custom error message better indicating that lazy initialization of pre-assigned instance slots isn't supported. Currently that case just lets the

[issue21145] Add the @cached_property decorator

Changes by Raymond Hettinger : -- nosy: +ncoghlan ___ Python tracker ___ ___

[issue21145] Add the @cached_property decorator

Carl Meyer added the comment: How do you propose that slots should be supported? Part of the value of cached_property is that cached access is a normal Python attribute access with no function call overhead. I am not interested in adding support for slots if it loses that benefit. I would not

[issue21145] Add the @cached_property decorator

Serhiy Storchaka added the comment: The decorator should support classes with __slots__. -- ___ Python tracker ___

[issue21145] Add the @cached_property decorator

Carl Meyer added the comment: Attaching a patch with the simplest version of cached_property (tehnique is not original, similar code is found in Django, Bottle, Flask, the cached_property lib, and many other places). -- components: +Library (Lib) keywords: +patch versions: +Python 3.7

[issue21145] Add the @cached_property decorator

Carl Meyer added the comment: I've used the cached_property pattern across many different projects, and never yet wanted a TTL. The simple "cache for the lifetime of the instance" behavior is easy to implement, easy to understand, and useful for a wide range of scenarios where instances are

[issue21145] Add the @cached_property decorator

STINNER Victor added the comment: "Most implementations these days support TTL because they require it." I used this pattern a lot in my old Hachoir project, but I never needed the TTL thing. In my case, data come from the disk and are never invalidated. Example with the description property:

[issue21145] Add the @cached_property decorator

Antoine Pitrou added the comment: I'm sure many people don't need a TTL on a cached property. Please stop arguing about that. -- ___ Python tracker ___

[issue21145] Add the @cached_property decorator

Omer Katz added the comment: In that case, most of the users won't use the standard library @cached_property anyway since they require it. On Mon, Feb 15, 2016, 19:51 Antoine Pitrou wrote: > > Antoine Pitrou added the comment: > > The TTL idea is completely outlandish

[issue21145] Add the @cached_property decorator

Antoine Pitrou added the comment: The TTL idea is completely outlandish in a general-purpose library. Let's keep things simple and not try to build a kitchen-sink decorator. -- ___ Python tracker

[issue21145] Add the @cached_property decorator

Omer Katz added the comment: Most implementations these days support TTL because they require it. The whole point is to remove the need to reimplement such basic functionality over and over. ‫בתאריך יום ב׳, 15 בפבר׳ 2016 ב-18:33 מאת ‪STINNER Victor‬‏ <‪ rep...@bugs.python.org‬‏>:‬ > > STINNER

[issue21145] Add the @cached_property decorator

STINNER Victor added the comment: I like the idea of an helper to build a property on-demand, but I dislike the TTL idea, it seems too specific. If you need TTL, implement your own decorator, or use a regular property and implement your own logic there. --

[issue21145] Add the @cached_property decorator

Serhiy Storchaka added the comment: May be. If somebody provide a patch. -- ___ Python tracker ___ ___

[issue21145] Add the @cached_property decorator

Omer Katz added the comment: Can we make this happen for 3.6? -- versions: +Python 3.6 -Python 3.5 ___ Python tracker ___

[issue21145] Add the @cached_property decorator

Daniel Greenfeld added the comment: For what it's worth, I just released cached-property on PyPI and someone suggested I join the discussion here. Package: https://pypi.python.org/pypi/cached-property Repo: https://github.com/pydanny/cached-property Notes: * 92% test coverage, albeit with a

[issue21145] Add the @cached_property decorator

Antoine Pitrou added the comment: Does this work for you? class X: ... @property ... @functools.lru_cache(None) ... def get_x(self): ... print(computing) ... return 5 ... x = X() x.get_x computing 5 x.get_x 5 -- nosy: +pitrou, serhiy.storchaka

[issue21145] Add the @cached_property decorator

Alex Gaynor added the comment: Will that implementation cause a memory leak? Won't the lru_cache have a dict mapping {self: result}, meaning that `self` will live forever, instead of storing result in self.__dict__, where the lifetimes are correct. -- nosy: +alex

[issue21145] Add the @cached_property decorator

Antoine Pitrou added the comment: Will that implementation cause a memory leak? Won't the lru_cache have a dict mapping {self: result}, meaning that `self` will live forever, instead of storing result in self.__dict__, where the lifetimes are correct. Oh, you're right. Sorry for the noise.

[issue21145] Add the @cached_property decorator

Omer Katz added the comment: The default implementation should simple: Once the property is accessed for the first time calculate the result and never calculate again. It's what both Django pip uses. You can add an optional TTL. There aren't any other features I can come up with that are common

[issue21145] Add the @cached_property decorator

Omer Katz added the comment: I just checked and werkzeug uses the same implementation as Django pip. 2014-04-06 14:31 GMT+04:00 Omer Katz rep...@bugs.python.org: Omer Katz added the comment: The default implementation should simple: Once the property is accessed for the first time

[issue21145] Add the @cached_property decorator

Changes by R. David Murray rdmur...@bitdance.com: -- nosy: +r.david.murray ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21145 ___ ___

[issue21145] Add the @cached_property decorator

Éric Araujo added the comment: It could make sense to add clean, working recipes to e.g. the functools documentation. The cached_property in the wiki uses a TTL, other like Pyramid’s reify decorator make properties that ensure the fget function is called only once per instance, and there may

[issue21145] Add the @cached_property decorator

New submission from Omer Katz: cached properties are widely used in various prominent Python projects such as Django and pip (and many more). They are not very complicated and there are proven implementation out there. Unfortunately there are way too many of them. This situation leads me to

[issue21145] Add the @cached_property decorator

Changes by STINNER Victor victor.stin...@gmail.com: -- nosy: +haypo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21145 ___ ___ Python-bugs-list

[issue21145] Add the @cached_property decorator

Madison May added the comment: There's currently an example of a cached property decorator implementation in the wiki, although it doesn't leverage functools: https://wiki.python.org/moin/PythonDecoratorLibrary#Cached_Properties -- nosy: +madison.may