Hi Andrey, On Thu, Oct 16, 2008 at 02:41:49PM +0200, Andrey Hristov wrote: > 1) Missing sequence number from the packet header. This sequence number > doesn't duplicate TCP functionality. During the development of the C > library for PHP I discovered server bugs where the sequence number > helped resolve them and was showing the client that there is something > wrong. libdrizzle won't have access to the TCP sequence number. Let me > show you an example of typical server bug: > COM_QUERY -> seq 1 > ERROR -> seq 2 > ERROR (this is the bug) -> seq 3 > > COM_QUERY -> seq 1 > > In this scenario you send 2 queries. The first generates an error, but > because of a server bug it there is second error. This has happened and > there is a potential to happen again. If there is no sequence numbering > then the second query will read the second error, which wasn't generated > by it and get confused, you will get out of sync. If the second query > has generated a result set, the third query will read it's result.
Ahh, ok. I was under the assumption that the sequence number was only there for proper ordering over network transports that did not guarantee order (like UDP). In the case you point out, it seems the right action to take is to actually xix the server bug generating duplicate errors, and making sure only a single error packet is sent (possibly with multiple errors included. I'd rather fix server bugs than introduce mechanisms into the protocol that don't need to be there. > One more thing in this regard. Every command should get a respond! In > the current MySQL C/S implementation sending of blobs (long_data) when > using PS doesn't generate a response. The client library just bombards > the server with data when finished doesn't wait for a response. A > problem can occur, if the server has too low max_packet_size. Then the > server generates an error, which is unexpected by the client. It will > have sequence number 2 and will be read by your next command. In this > the client gets wrong response and can't detect. So, either everything > has a response, or start the sequence number from a random number and > increment it. First is better. With the exception of the client handshake, every command will get a response. Out of band errors will also be allowed, so a client can abort sending of huge blobs when an error has been detected early (say if the server preparses some and detects auth is needed). > 2) Authentication > Your description mentions that there will be late authentication still > during handshake the proposed protocal specifies what authentication > protocol (plugin) should be used. I think this should go to the > authentication response after a command. This is more flexible. During the handshake, the server announces what auth types it supports, and the client reports with a single preferred method. I'm not sure what flexibitilty we'll really gain by pushing the supported-auth types into the command response. This also introduces a new RTT, since the client would need to respond again with the auth type to use, and then finally the server pluging to initiate the auth (such as giving the scramble bytes). The method use in the current draft allows the server pluging to send initial infomation it in the same network packet as the command response. Do you have an example of where pushing this functionality into the command response would be helpful? All I can think of is different auth types per DB or table, but I'm not sure if we'll support this. > 3) There is missing change user functionality - re-authentication My thoughts here were the server would ask if the current user doesn't have the proper credential, but I can see cases where you may want to change user even if the permissions don't matter (logging or reporting purposes). I'll add a new command to force a new auth response and plugin exchange. > 4) Rows affected/processed... > "OK > -- > > If an OK response is non-zero, this represents the number of fields in > the response. After the value, a list of parameters may follow, and the > end is indicated by byte with value '254'. Possible OK parameters are: > > 0 ROWS_EFFECTED - Length-encoded count of rows effected. > 1 ROWS_SCANNED - Length-encoded count of rows scanned. > 2 WARNINGS - Length-encoded count of warnings encountered. > 3 INSERT_ID - Last insert ID. > " > > There is a reason that this data appears in the MySQL C/S protocol after > the result set and not before it. Because the server streams the data > from disk/memory to the network. Subqueries sometimes are materialized > but don't expect everything to be materialized so the server knows the > values before starting with sending the data. Good point. I'll add the ability to send a summary packet in the end with the same set of result parameters. This will allow them to be omitted in the beginning and given at the end instead. > 5) Value 251 is used in the text protocol to mark a NULL value in the > row. It consumes 1 byte and this can be optimized. The idea comes from > MySQL's binary C/S protocol. Use a NULL-bitmap. You can pack up to 8 > columns in the single byte. Actually, it's 6 columns for the first byte > and 8 columns for every next. I can even propose using the binary > encoding from the protocol, as it can be more efficient - saves CPU > cycles on the server by not converting ints/floats to strings, saves CPU > cycles on the client by not letting the client convert strings to > ints/floats. I'll have to think about this a little more. The bit map would only really be useful when there are many adjacent NULL fields, so this is an optimized case for a specific application usage. One of the ways to extend the protocol I didn't mention is compression, so if space is really an issue you could always enable a compression scheme (pick one with the right space/time tradeoff for your usage). We may still want to do something like a NULL bitfield in the core protocol if it's common enough. > You mention configurability of which encoding to be used, to be set > during handshake. IMO, it's better to add a flag (a byte) in the > COM_QUERY response which format was used to encode the data. Thus, one > can switch between encoding implementations with an SQL command and > doesn't need to track what was lately set - makes the work on the row > decoding level quite easier. This is important for scripting languages, > because there the driver developer doesn't need to check every query the > user does, whether it switches the encoding - fool-proof. Ok, it does make a bit more sense to push this option to the command level rather than entire connection level. Some clients may want to switch between the two depending on the query (and not have to reconnect). I'll add this support. Thanks for the detailed response! -Eric _______________________________________________ Mailing list: https://launchpad.net/~drizzle-discuss Post to : [email protected] Unsubscribe : https://launchpad.net/~drizzle-discuss More help : https://help.launchpad.net/ListHelp
