Stephen Deasey wrote:
Hit rate is not necessarily the best measure of effectiveness. Perhaps it's more expensive to fill one cache than it is to fill another? A single, shared LRU seemed like the simplest solution. Ideas appreciated.
What if 'ns_cache eval' stored with the cached value an estimate of how much work (i.e., time) it took to create the cache entry, and used that as a hint for expiring or flushing values? An operation that is used 1/10 as often but takes 50x longer to compute may be worth keeping around longer.
And as long as I'm musing on caching, 2 situations dealing with expensive cache calculations come to mind. One is where there is no current value for an entry and several threads request it simultaneously (or it is requested multiple times before the first request has time to complete evaluation of the script). Do the second and subsequent accesses start their own evaluation of the script, or do they block until the first completes and then all return the single value? Without digging into the code or running tests, my guess is that this works correctly, since the key should be locked with a mutex and reads would block the thread until the mutex is released.
A second situation is where there is an expired entry and multiple threads access it at the same time. It is probably handled as above, as if there was no entry. But it might be desirable in certain situations to return a stale entry immediately and start a separate thread to run the script.
These solutions may be too clever for their own good and apply only to a very few marginal cases.
-J
