Billion items can be represented using about 30 bits. Using binary protocol you can have 4 byte keys assuming some fantastic hash function or a bad one if collisions is not a problem but at client end not in memcached. With ascii protocol the number of bytes would probably double.
Thanks Rohit Karlupia On 24 Oct 2011 00:26, "dormando" <[email protected]> wrote: > > You have to know the key to map it to a server and RAM location, but I > > do not see any reason why you should have to store it. > > The only explanation seems to distinguish values after a hash > > collision, since you can not even iterate over the keys. > > I might provide it as an engine at some point, but in order to do that you > have to change the hash table somewhat significantly, and you lose enough > bytes that it's not worth while: > > 1) you have to expand the hash table size enough so there're generally few > collissions, and when there are you can store in the bucket to the right > of the intended one. Right now I think the hash table is 1/3rd the total > item store size as a performance tradeoff. > > 2) You have to store the hashed value with the items, so when pulling them > off of the LRU, you can look up their hash table entry and remove them. > Presently we do that by recalculating the hash from the key. > > So if your keys are already short it'd be a toss-up for the byte tradeoff. > If you generally have long keys, you should really just hash them in the > client anyway, as that saves you in a lot of other ways :P > > > I'm using the binary protocol. The problem is that both md5 and sha1 > > return \n, \r and spaces in the key, which leads to this message: "Key > > contains invalid characters". Is there any way to use a byte array as > > key instead of a string? > > Are you absolutely sure? Which client is this? > > With the binary protocol, keys are binary blobs and it doesn't matter > what's in them. There may be an option you have to swith off in the > client, or the client may not actually be using binprot (run memcached > with -vvv to see what it thinks of your connecting clients). > > -Dormando >
