On Fri, Mar 15, 2013 at 4:07 AM, Jens Geyer <[email protected]> wrote: > Hi, > > I like the idea of the data types getting promoted to bigger ones on read, > but I'm still afraid it won't work. Lets assume a service like this: > > struct foo { > 1: i16 bignumber, > 2: i16 anotherone, > 3: i16 third > } > > service bar { > foo Something( 1: foo input) > } > > Now, after some time, you find out, that i16 is way too small, so you change > foo{} to hold all i32's. > > First, the real issue is, when not all clients and servers are under your > control and can be updated at once, it will break. The generated code holds > declarations for all variables, so it can't be promoted on the fly only as > far as the current data structures allow this. > > But OTOH the idea has some good aspects, since it brings in the idea that > the size of the variable must not necessarily be equal to the amount of the > bytes on the wire, e.g. a int32 = 0x00001234 could be read/written as i16 > without losing information. The only thing that may be affected here is the > performance, because the code has to check for appropriate write and read > sizes.
I think TCompactProtocol uses var length ints, achieving something similar to what you mentioned. Not that this would help w/ the original issue. > > Jens > > > -----Ursprüngliche Nachricht----- From: Will Pierce > Sent: Thursday, March 14, 2013 10:47 PM > To: [email protected] > Subject: Re: updating to longer ints > > > Definitely not an easy problem. On the wire these will both encode as the > same field ID (1) and a type ID of 6 for the i16 or a type ID of 8 for the > i32. Most (all?) thrift language bindings check to ensure the type ID for > a field matches the expected type ID for that particular field, silently > skipping past any field with mismatched type IDs. > > George Chung's suggestion is the safest/cleanest approach. > > I had exactly this same problem where I needed to expand from i32 to i64. > It was a "flag day" cutover for me, since I had control of all the clients > and servers. But I can see how you frequently can't upgrade all the client > code out in the wild. (Or data files, if these are persisted.) > > Rather than try to solve the broader problem of versioning, maybe we should > consider adding optional behavior to the decoder to permit > *type-promotion*when the field's on-wire type can be safely decoded > into the expected type? > > Provided that the expected type can faithfully represent all the values of > the wire type, it seems like a way to follow the "TCP" pattern of being > permissive in what you receive, but strict in what you send. > > In terms of thrift TTypes, we could allow a decoder to promote: > bool > i08 > i16 > i32 > (double or i64) > > This wouldn't solve every problem - like sending a response struct with an > i32 field to a client than expects i16. No go there. This would be only > useful in cases of having to read client input from a mixed bag of old and > new clients, where the server response struct doesn't need any wider > fields. I'm not sure if that is your use-case... > > Those using thrift for serialization to/from disk, I would imagine > type-promotion could save a lot of effort in re-encoding any old files that > use narrower numeric types than newer files... > > Thoughts?
