Attached is the most current documentation of the binary protocol after
hammering through more details at the Yahoo! hosted hackathon today.
I have not gone through the details about the semantics of the
command-specific extra values, except to copy forward text from the
version 1 document from the July hackathon hosted at facebook.
Please review and post comments.
Aaron
Notes regarding the proposed binary protocol from Yahoo's hosted
memcached hackathon on 2007-12-13:
This document current as of 2:45 AM on 2007-12-14.
Bits run across the top. Period (.) represent the
start of each byte (bit 0 in that byte, so to speak).
The characters -, +, | are used for Ascii art.
Number of bits in each field are in parens.
Numbers on the left side are helpful for counting
four byte word lines in the packet.
All numbers are in network byte order (big endian).
Request header:
+.1234567.1234567.1234567.1234567+
1 |Mag (8) Cmd (8) Key len (16) |
2 |Len (8) Type(8) Reserved (16) |
3 |Total body length (32) |
4 |Opaque key (32) |
+.1234567.1234567.1234567.1234567+
Response header:
+.1234567.1234567.1234567.1234567+
1 |Mag (8) Cmd (8) Stat(8) Resvd(8)| ***FIXME: see below
2 |Len (8) Type(8) Reserved(16) |
3 |Total body length (32) |
4 |Opaque key (32) |
+.1234567.1234567.1234567.1234567+
Brevity:
The short keywords in the header format expand as follows:
Mag:
magic number
Cmd:
command code
Key length:
length in bytes of the text key that
follows the command extras.
Stat:
status of the response (non-zero on error).
Resvd:
FIXME: Dustin, please remind what this is?
Len:
length in bytes of the command extras.
Type:
reserved for future use (Sean is using this soon).
Reserved:
really reserved for future use (up for grabs).
Total body length:
length of extra + key + value
Opaque key:
will be copied back to you in the response.
General format of a packet:
+.1234567.1234567.1234567.1234567+
1 |HEADER (16 bytes, per above) |
2 | |
3 | |
4 | |
+.1234567.1234567.1234567.1234567+
|Command-specific stuff |
| (note length in byte 5) |
+.1234567.1234567.1234567.1234567+
|Key |
| (note length in bytes 3-4) |
| (never present in responses) |
+.1234567.1234567.1234567.1234567+
|Value |
| (note length of bytes 9-12, |
| minus extra length, |
| minus key length) |
+.1234567.1234567.1234567.1234567+
Magic:
* Magic byte / version. For each version of the protocol, we'll use
a different request/reponse value pair. This is useful for protocol
analyzers to know what a packet is in isolation from which direction
it is moving.
* The version should hopefully correspond only to different meanings
of the command byte. In an ideal world, we will not change the
header format. As reserved bytes are given defined meaning, the
protocol version / magic byte values should be incremented.
* Magic byte values will begin at 0x80 and go up from there.
Commands:
get - single key get (no more multi-get; clients should pipeline)
getq - like get, but quiet. that is, no cache miss, return nothing.
Note: you're not guaranteed a response to a getq cache hit until
you send a non-getq command later, which uncorks the
server which bundles up IOs to send to the client in one go.
Note: clients should implement multi-get (still important for
reducing network roundtrips!) as n pipelined requests, the
first n-1 being getq, the last being a regular
get. that way you're guaranteed to get a response, and
you know when the server's done. you can also do the naive
thing and send n pipelined gets, but then you could potentially
get back a lot of "NOT_FOUND!" error code packets.
alternatively, you can send 'n' getqs, followed by an 'echo'
or 'noop' command.
+.1234567.1234567.1234567.1234567+
1 |Flags (32) |
2 |CAS ID (64) |
3 | |
+.1234567.1234567.1234567.1234567+
delete
When allows you to 'reserve' a key. When 'when' is set
for, say, ten seconds in the future, all 'set's on that
key will fail until ten seconds from now.
+.1234567.1234567.1234567.1234567+
1 |When (32) |
+.1234567.1234567.1234567.1234567+
set/add/replace
cmd-specific fixed-width fields for set/add/replace:
* 4 byte expiration time
* 4 byte flags
(the 4 byte length is inferred from the total body length,
subtracting (keylen + body length))
+.1234567.1234567.1234567.1234567+
1 |Flags (32) |
2 |Expiration (32) |
3 |CAS ID (64) |
4 | |
+.1234567.1234567.1234567.1234567+
noop
Used as a keep alive. Flushes outstanding getq's.
Should not have any protocol specific gunk.
Must not have a key.
incr/decr
+.1234567.1234567.1234567.1234567+
1 |Amount (64) |
2 | |
3 |Initial value (64) |
4 | |
5 |Expiration (32) |
+.1234567.1234567.1234567.1234567+
