On Tuesday, October 18, 2011 7:54:33 PM UTC-7, Dormando wrote: > > > We had previously talked about an even tighter binary protocol, but > perhaps harder to generalize a parser around. ï¿œThis doesn't seem different > > enough from the existing binary protocol to warrant introducing an > incompatibility. > > I honestly can't remember what else was removed in our old discussions. > Old man is old. Something about the "reserved bytes" turning into a longer > flag set maybe? This proposal just drops those bytes. > > What I may have mentioned was that uh... size length encoding, similar to > what MySQL uses. So a key length of < 252 bytes would be a 1b length. a > body length of just under 2^16 would be 3 bytes (253 as a flag + 2 bytes), > and over would be 4 bytes (254 as a flag + 3 bytes). > > Could be nice for getting CAS down in size, but you end up inflating an > extra byte for larger values. >
I don't have the notes from that discussion, but there was the dream of the "five byte get" A byte of magic, a byte of flags describing the parts, a byte of opcode, and a byte of key length, then a key, "a" The flags would include things like quiet, CAS?, keylenlen, etc... It's harder to work with, but really tight. > > * I like the idea of a separate quiet flag from the opcode (but will > happily settle for having the bit in a consistent position) > > Using the last spare flag in the magic byte? Even if we bump up a whole > byte for flags, that only gives one extra. Originally I had it listed in > the magic byte, but moved it to the opcode bit to free up a flag. I'd > accept a strong argument to the contrary, though. > My only argument is wanting more opcodes. :) > > * Tap has a variable length extras, but we could possibly just push that > into the body. > > I'm too blind at a glance to see where that is in the wiki page you wrote. > Would it be a huge pain to change? > I think we could move it into the body. Trond can insert an opinion here. > I didn't spend a ton of time re-aligning the bytes after removing a chunk > > of them. Given the flags you send, CAS may not end up fully aligned in the > header anyway. That's not necessarily how the API end has to implement it > however, that's just how it has to be read off the wire. > Yeah, that's the problem with packing the stuff tighter. You have to move stuff around more to get alignment. > > * I like the ASCII stuff, but I would like to see more stuff around quiet > that make it harder to get wrong. > > Can you expand on this? The way I tried to write it was direct compat with > the way binprot is handled. ie; you can shove commands down the barrel and > read them back asynchronously, matching up opaques or keys to specific > errors. See the (admittedly barely notated) "response" section for ASCII. > It didn't look like opaque was required for quiet commands in ASCII. I think that's the big difference. > > Clients can ignore (or log or whatever) unsolicited messages they don't > know how to handle, but some operations get *very* simple (observing > > internal memory restructuring, "jobs" over keys, persistence operations, > proxy operations, etc...) > > How does binprot stop you from doing this presently (besides saying you > can't)? > I was thinking about this afterwards and it really seems like a good solution is to just have three magics: * Request Magic * Response Magic * Unsolicited Response Magic Then you can receive unsolicited responses at any point without having to worry about confusing it with normal responses. Does this makes sense? >
