Hi Vicente,
Vicente Aguilar wrote:
Hi
At work we're setting up a multi-layered cluster with a 1st line of
web/proxy/cache servers (nginx+memcached) and several layers of
PHP-FPM and Tomcat application servers. At the moment we're using
memcached just to cache raw HTML pages, not serialized objects: upon a
request nginx checks if that URL (key) is on memcached and then serves
it, else it proxies the request to the appropriate app server which
serves it and saves it to memcached. Pretty basic memcached usage.
The problem is that we can't get the PHP, Java and nginx
implementations to work together, the PHP and Java consistent hashing
algorithms don't seem to match.
We (Dustin Sallings and Steve Yen mainly) did a lot of testing in
spymemcached to ensure the hashing was the same between it and libketama
in the most recent release. There is a very large test of these in
spymemcached. In the process a bug was found and fixed, but it was a
very minor bug.
It was compared to what libmemcached is doing with it's hashing and it's
been well verified. More below...
We're using:
- nginx: http://wiki.nginx.org/NginxHttpUpstreamConsistentHash
<http://www.google.com/url?sa=D&q=http://wiki.nginx.org/NginxHttpUpstreamConsistentHash&usg=AFQjCNHc6onwPil5pzVr3imuikObHBbz9w>
- PHP: PECL:memcache http://pecl.php.net/package/memcache
- Java: spymemcached http://code.google.com/p/spymemcached/
Both the nginx and PHP implementations match, any HTML representation
saved by PHP is retrieved by nginx from the same memcached server; but
spymemcached's implementation seems to be different and with 2
memcached server only matches 50% of the time, so it's a completely
different algorithm and the matches are random. We're using the nginx
patch and PECL:memcached with their default configuration, and as for
spymemcached we've tried the NATIVE_HASH, CRC32_HASH and KETAMA_HASH
algorithms but none of them matches nginx' and PHP's hashing.
Anybody else here is using memcached in Java and PHP and got them to
work together? I've searched the list and this question has been
raised several times but haven't find any definitive answer, just the
obvious "different libraries may use different hashing
implementations". We don't mind switching to other libraries.
Assuming you're using the latest spymemcached (which was released some
time ago) and a recent pecl/memcached, at least those two should work
together. The PHP level wasn't, but the libmemcached ketama and
spymemcached ketama are well tested.
With only two servers, nearly anything could make it be 50% off but it
almost sounds like one client is seeing one of the servers as down. In
a ketama world, that'd mean put all of it on the remaining up servers
(the other one in your case).
The only other thing to be careful with is perhaps the key (which seems
to be a URL in your case?) is getting truncated or treated differently
in different places?
Sorry I don't have any clear answers here, but I can say it's been
tested and SHOULD work.
- Matt
