dormando wrote:
Dustin Sallings wrote:
On Dec 6, 2007, at 19:03, dormando wrote:
- We discussed the alternative method of wrapping the real expiration
time inside your serialized object, and using that instead. We think
that works out better for all invovled :)
The thing I've mentioned a couple of times, but have put no effort
into whatsoever is the ability to subscribe to an invalidation stream
somehow. It'd be pretty awesome to simply be notified when things
happen that invalidate keys.
There are obviously a few large holes that come about when one goes
to implement such an idea (it shouldn't weigh down mutations, and
there's the question of how to be notified when something just
expires). I suppose the expiration time could be skipped if the
second-level cache *also* tracked the times.
It'd be a hard thing to get right, but it's the right way to do a
second-level cache.
Trying to think of how to use this... If something's expired in my
cache, there're good odds it'll be requested in that same second, or
very nearterm. It's more efficient to just let it expire and recache on
its own.
What're some problems this could help resoundly solve? My lack of
creativity is embarassing.
-Dormando
I implemented this by wrapping the time in the data, as suggested above.
The reason I did this, is if something is expensive to generate and very
popular, and updates relatively infrequently or updates often but can be
slightly out of date, I'd like to regenerate it once in a while before
it actually expires. By including the expiry time with the data, I can
tell when it's going to expire, and so a couple seconds before that, I
get a lock, regenerate it, and store it. If I didn't get a lock, that
means a different process is already regenerating it, so I can just use
the old version. This is much nicer than creating a cron script to run
every minute, or whatever frequency I need, and is fairly fault
tolerant. It means that when it does hit that expiry time it is only
regenerated once, not by every process that gets that request between
the time of expiry and the time of it getting back into memcache.
By storing the expiry time in the data, I can also have my L2 cache
cache it locally with the same expiry so that when it gets updated to
memcache, it also will propagate to the L2 cache quickly.
This setup is not good for data that needs to be expired explicitly, but
that could be added later.
Having the expiry time returned explicitly instead of part of the data
does make things slightly easier, but otherwise isn't much of an advantage.
Timo
