On Sat, Mar 8, 2008 at 11:40 PM, Henrik Schröder <[EMAIL PROTECTED]> wrote:
> Hi everyone, > > I saw the thread on ketama in the java client, and it made me wonder what > the state of interoperability is among the different clients? If I > understand it right, the PECL PHP Client is as close to a standard as you > get, since it's the most commonly used one. As for consistent hashing, it > seems like the libketama thingie is the standard there, and that is some > sort of C library? > > When we developed the BeITMemcached client, we really didn't care about > interoperability since we only work in C#, and the majority of the objects > we cache are serialized .Net objects, and those are pretty useless in other > languages. But I'm kinda curious, how common is it with a mixed environment > where you have two different clients talking to the same memcached servers? > > In any way, I figured it would be *nice* if our client also behaved as > close to the standard as possible, but I'm wondering what this standard is, > with regards to server selection using consistent hashing. If you use the > simple server selection, it's pretty straightforward. Server = > serverlist[hash(key) % serverlist.Length], and the only thing you can > modify is the hashing algorithm used. But if you use the "ketama" consistent > hashing selection, then you suddenly have more parameters, and it seems like > most clients do it their own way? > > When placing the servers on the continuum, you have to choose how many > times each server appears, which hashing algorithm you use to generate the > places, and which scheme you use to get each successive point for a given > server. For BeITMemcached, we chose a modified FNV1a hash, we hardcoded it > to generate 100 hashes, and each successive hash is created by iteratively > hashing the server name. I know that other clients append some characters to > the server name, and that they use some other hashing algorithm, and I > couldn't find anything authoritative saying how you should do it. > > So, for those of you who maintain other clients, how important is > interoperability to the people that use them? Have you been asked to > consider it? Is it worth changing our client so that you could get > consistent server selection between different clients? Its really important to me as my cache stores data for both Java and C# clients :) I've been working hard to ensure that at least 1 Java and 1 C# client are fully interoperable, both in terms of the data storage and in terms of the node location algorithms. You can find my notes so far on the memcached wiki: http://www.eu.socialtext.net/memcached/index.cgi?cross_platform_serialisation_support Interestingly on re-reading it, I notice that I didn't write down my thoughts on the node locator compatibility, but in discussions with the two client authors described on that page, it was generally agreed that 'libketama' was a reasonable 'gold standard' Thanks - Ciaran > > > > /Henrik Schröder > > > -- - Ciaran
