On Thu, Oct 16, 2008 at 01:23:30AM -0700, Eric Day wrote:
> http://drizzle.wikia.com/wiki/New_Protocol
I have some ideas/suggestions:
have you looked at NDB Protocol6? see
storage/ndb/src/common/transporter/TransporterInternalDefinitions.hpp in
a MySQL tree. One interesting thing is the (optional) trace value. This
can be used to have signal id's passed across the wire so you can (from
a dump of the signals being sent) see which signals are a reply to what.
e.g.
server client
ROW(1) ------------>
ROW(2)
ROW(3)
<------------ STOP(1)
ROW(4) ------------>
STOPPED(5) ------------>
which would let you work out from sniffing the wire that the STOP was
sent because of the ROW(1) signal, not in response to ROW(2) or ROW(3).
Useful in tracking down bugs (e.g. if we had a bug where receiving rows
after we'd sent a stop caused a crash).
- command id: we use 'userReference' in NDB for this term.... useful for
indicating that the user is 100% responsible for it. command id sounds a
bit like it may be server allocated (even though it's not).
- command id length. only 2 bytes seems small.
Currently
(http://jonasoreland.blogspot.com/2008/11/500k-reads-per-second-on-1-datanode.html)
8 threads and 512 simultaneous txns is a small benchmark for NDB.
So if we were going to multiplex the command ids between multiple
threads (keeping at 512 concurrent), we'd run into limits here at a mere
128 threads (no doubt we'll have this many cores in a sparc chip soon).
I suggest a bump to full 32bit. This should be enough for the next 10
years or so...
- command/result code. 1 byte seems conservative...
We already have 8 listed there... and i can easily think of a lot
more:
- key lookup
- scan
- stop sending me rows
Or is these intended to be in the packet parameters?
If so, this means command id should be in there, not in outer part.
I'm thinking of client saying something like "here are 100 key lookups
and 100 callback functions". When ack comes on that query, client lib
should call that callback function. Client could have command id 0-100
and have an array of things to lookup. so when executing replies, it
just looks it up and calls.
FIELD_ENCODING
- I have a feeling this should be per-column...
let's say we add 128bit integer support to the server.
clients will know binary form of 64bit integer.
(okay... now i'm waiting for Jim to pipe up on the Falcon number
storage thingy :)
In this situation, client only needs textual representation of the
128bit integer.
Could go like this:
- FIELD_ENCODING: native to field_type id 42 (which is "64bit int")
(plus some checksum of all the previous ones)
and server would respond either with:
- umm... my field_type id 42 isn't the same as my 42 (or checksum
dosen't match)
- then do backwards search until they both can agree on
what they both know the binary encoding for.
(this solves the problem of having forks/trees with different
sets of fields loaded)
- proceeding as normal, going "okay, for everything less than
type 42, i'll send binary."
This means that correct client code wanting the binary representation
will have to have fallback. e.g.
if(drizzle_get_field_int64(&resultset, 1, &my64int)==DRIZZLE_ER_NO_BINARY)
{
real_str_length= drizzle_get_field_string(&resultset, 1,str,len);
my64int= strtoll(str, str+real_str_length, 10);
}
This means that if you update libdrizzle so that it does understand new
types, the get_field_string will still work (as field to string
conversion should be in libdrizzle or if talking to old server, done in
server).
I'm also thinking that FIELD_ENCODING should only be sent once per
result set.
--
Stewart Smith
_______________________________________________
Mailing list: https://launchpad.net/~drizzle-discuss
Post to : [email protected]
Unsubscribe : https://launchpad.net/~drizzle-discuss
More help : https://help.launchpad.net/ListHelp
