Memcached "get_or_lock" and defer "delete" functions Proposal
ABSTRACT
--------
On busiest web sites who depend on memcache technology, when cache
expired or deleted, they should face database query flood. This is the
Concurrent-Cache-Miss-Renew problem. This proposal provide a Defer-
Expiration mechanism to solve the problem.
This proposal also recall the defer "delete" function back to solve
frequent delete problem.
MOTIVATION
----------
On our web site, always we use following piece code to acting with
memcache:
<code lang="php">
$val = $memcache->get($key);
if (!$val) {
$val = $db->query($sql);
$memcache->set($key, $val, $expire);
}
</code>
If $sql is large time consumed, then when cache expired on high visits
period, we should face database load too high problem.
Why? Because $sql is complicate, then the database query should need a
bit long time. Before the database query complete, there should be a
lot of concurrent requests got cache-miss and do same cache-renew
process. Too many concurrent and complicate $sql query should slow-
down the database execution and let more requests fall in cache-miss
and cache-renew hell. This is a positive feedback until the cache be
renewed lastly.
In our practice, this problem led our site unstable on high visits
period randomly. And, when it happens, always need several seconds to
get back stable.
About defer `delete` function, when we frequent update database and
delete cache entry (e.g., delete entry 20 times in 1 second), we will
got lots cache-miss and enhanced the above (Concurrent-Cache-Miss-
Renew) problem.
SOLUTION
--------
We patched memcached to add "get_or_lock" instruction to provide Defer-
Expiration function. Sample code first:
<code lang="php">
$val = $memcache->getOrLock($key);
if (!$val) {
$val = $db->query($sql);
$memcache->set($key, $val, $expire);
}
</code>
When use "get_or_lock" and cache-hit, no different behaviour with
current memcached's "get" implements. But when cache-miss, and expired
less than specific defer-expire time, e.g. 3 seconds (configable),
then first two "get_or_lock" requests will got cache-miss and others
got the staled value per second. When cache expired eq or above 3
seconds, the behaviour becames same as classic "get" again. That's
all.
Where the "get_or_lock"'s behaviour is unacceptable, we can simply
switch back to use "get". No comflicts with each other.
About defer `delete` function, Sample code first too:
<code lang="php">
$memcache->delete($key, $hold);
</code>
This is only recalled the deprecated memcached function but use
reliable and consist expire model the "get_or_lock" enforced.
Please note, on memcached text protocol, "get_or_lock"'s command is
'lget', 'lgets', etc. On binary protocol, I use
PROTOCOL_BINARY_CMD_LOCKBIT (0x80) to indicates locked version 'get',
'gets' and 'getk'.
CONCLUSION
----------
We use "get_or_lock" combine with defer "delete" gains performace and
stablity on our site on high visits period. Adopt these modifications
makes our site's HTTP 502 statistics down from 12000~14000 to 50~100
per week.
