[issue18577] lru_cache enhancement: lru_timestamp helper function
Ben Hoyt added the comment: I really like this idea (and am needing this functionality), but I don't think this API (or implementation) is very nice: 1) It means you have to change your function signature to use the timeout feature. 2) Specifying the interval in minutes seems odd (most similar timeouts in Python are specified in seconds). I would love to see an optional timeout=seconds keyword arg to the lru_cache() decorator, or some other way to support this. Raymond, what do you think would be the simplest way to hook this in? One way I think would be nice (and also support other neat things) is to allow you to specify the dict-like object that's used for the cache (defaults to dict, of course). So the lru_cache signature would change to: def lru_cache(maxsize=100, typed=False, cache=None): ... From looking at the source, cache would need to support these methods: get, clear, __setitem__, __contains__, __len__, __delitem__ Would this just work? Or could there be a race condition if __contains__ (key in cache) returned True but then cache.get(key) returned False a bit later? In any case, this seems nice and general to me, and would mean you could implement a simple ExpiringDict() and then pass that as cache=ExpiringDict(expiry_time). Thoughts? -- nosy: +benhoyt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18577 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18577] lru_cache enhancement: lru_timestamp helper function
Peter Santoro added the comment: As requested, I published this for review on http://code.activestate.com/recipes/578817-lru_timestamp-cache-entry-aging-for-functoolslru_c/ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18577 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18577] lru_cache enhancement: lru_timestamp helper function
Raymond Hettinger added the comment: Please publish this outside the standard library so it can mature and get user feedback. I think it would be premature to add it right now. The subject of cache entry invalidation or expiration is broad. I'm not sure this is the best way to do it. -- resolution: - rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18577 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18577] lru_cache enhancement: lru_timestamp helper function
Raymond Hettinger added the comment: This is a pretty interesting idea. Ideally, it would be great if it could be a published as a recipe somewhere so that people could experiment with the API and try out variations. If there were good uptake by users, it would help justify a proposal to be included in the standard library. -- priority: normal - low ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18577 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18577] lru_cache enhancement: lru_timestamp helper function
Peter Santoro added the comment: I updated my proposed lru_timestamp function with the following change: 1) raise TypeError instead of ValueError -- Added file: http://bugs.python.org/file31079/lru.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18577 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18577] lru_cache enhancement: lru_timestamp helper function
New submission from Peter Santoro:
The attached proposed lru_timestamp function provides developers with more
control over how often lru_cache entries are refreshed. Doc string follows:
def lru_timestamp(refresh_interval=60):
Return a timestamp string for @lru_cache decorated functions.
The returned timestamp is used as the value of an extra parameter
to @lru_cache decorated functions, allowing for more control over
how often cache entries are refreshed. The lru_timestamp function
should be called with the same refresh_interval value for a given
lru_cache decorated function.
Positional arguments:
refresh_interval -- 1-1440 minutes (default 60) as int or float
Rationale:
Some functions have input parameters that rarely change, but yet return
different results over time. It would be nice to have a ready-made solution to
force lru_cache entries to be refreshed at specified time intervals.
An common example is using a stable userid to read user information from a
database. By itself, the lru_cache decorator can be used to cache the user
information and prevent unnecessary i/o. However, if a given user's
information is updated in the database, but the associated lru_cache entry has
not yet been discarded, the application will be using stale data. The
lru_timestamp function is a simple, ready-made helper function that gives the
developer more control over the age of lru_cache entries in such situations.
Sample usage:
@functools.lru_cache()
def user_info(userid, timestamp):
# expensive database i/o, but value changes over time
# the timestamp parameter is normally not used, it is
# for the benefit of the @lru_cache decorator
pass
# read user info from database, if not in cache or
# older than 120 minutes
info = user_info('johndoe', functools.lru_timestamp(120))
--
components: Library (Lib)
files: lru.py
messages: 193820
nosy: pe...@psantoro.net
priority: normal
severity: normal
status: open
title: lru_cache enhancement: lru_timestamp helper function
type: enhancement
versions: Python 3.3
Added file: http://bugs.python.org/file31063/lru.py
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18577
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18577] lru_cache enhancement: lru_timestamp helper function
Peter Santoro added the comment: I updated my proposed lru_timestamp function with the following changes: 1) restricted refresh_interval to int type 2) updated doc string Updated doc string follows: def lru_timestamp(refresh_interval=60): Return a timestamp string for @lru_cache decorated functions. The returned timestamp is used as the value of an extra parameter to @lru_cache decorated functions, allowing for more control over how often cache entries are refreshed. The lru_timestamp function should be called with the same refresh_interval value for a given @lru_cache decorated function. The returned timestamp is for the benefit of the @lru_cache decorator and is normally not used by the decorated function. Positional arguments: refresh_interval -- in minutes (default 60), values less than 1 are coerced to 1, values more than 1440 are coerced to 1440 -- Added file: http://bugs.python.org/file31064/lru.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18577 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18577] lru_cache enhancement: lru_timestamp helper function
Changes by Raymond Hettinger raymond.hettin...@gmail.com: -- assignee: - rhettinger nosy: +rhettinger versions: +Python 3.4 -Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18577 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
