On Thu, Oct 7, 2010 at 11:18 PM, Jason Hsueh <[email protected]> wrote:
> > > On Thu, Oct 7, 2010 at 7:02 PM, Igor Gatis <[email protected]> wrote: > >> >> >> On Thu, Oct 7, 2010 at 6:19 PM, Jason Hsueh <[email protected]> wrote: >> >>> Indeed, maps have been brought up repeatedly. I forget the current state >>> of the discussion, but I think it's generally agreed that it would be a good >>> thing to add; it's just a matter of how to implement it (and finding the >>> time to do it). >>> >>> A couple of the major issues: >>> - backward compatibility with existing repeated fields >>> >> >> I believe this has been addressed by the proposal I have made. Maps a >> repeated fields. So they can be exposed as list of pairs (string, message). >> > > For backwards compatibility, the idea was to allow current repeated message > fields to be converted to maps. > I see. I don't think that should drive the new design though. The design I proposed requires a wrapper message. So if your declared repeated field is of type Foo. Internally, protobuf compiler would generate a new message, like the following: message KeyValuePair_of_Foo { required string key = 1; optional Foo foo = 2; } In order to migrate a repeated field into a mapped field one should declare a new field which is the mapped one, and move all items from the repeated field to the mapped field. This is required because the wire types are different. > You might expose these as pairs (string, message), or just as regular > message accessors, except with a key parameter instead of an index. > I see the value of exposing the internal message type (access key and values without doing several hash lookups). But that can be achieved easily by a simple wrapper just like STL's pair<string,*> object, or Java's Map.Entry, and Pythons (key, value) tuple. > On the wire though, the map field would be serialized as the messages only. > Keys are not serialized separately; they're just a special field within the > message, as in the example in the .proto file. That leads to ... > Well, I believe the migration scheme I described above is just much simpler. One thing to keep in mind: protobuf compiler is already too complicated. The mechanism you described adds more complexity to it. Every time you think of adding complexity to, remember to multiply it by the number of programming languages available out there, or even worse, multiply it by the number of third party implementations. Complexity is worth adding when tons of people get benefit of it. The backward compatibility stuff will benefit, say, less than 100 people out there, IMHO. The migration mechanism I described is way simpler and does not add complexity. > > >> >> >>> - ensuring that the client can't change the map key in the mapped value >>> without also updating the map structure. >>> >> >> Could you provide an example? >> > > In the example from the .proto, suppose we generate key-based accessors for > the items map field, and internally the field is represented as a > hash_map<string, Item>. > > Item* mutable_items(const string& name); > const Item& items(const string& name); > > A client could do something like: > const string key = "foo"; > foo.mutable_items(key)->set_name("bar"); > > i.e. its logical key (and its key when serialized and reparsed) is "bar" > but the hash_map points to it using the old key "foo". > You see. This is not possible in the proposal I've made. The key is separated from the object. The (String,Message) message never gets exposed. It is not possible to modify it through the generated API. > > >> >> >>> >>> >>> On Wed, Oct 6, 2010 at 11:17 AM, Igor Gatis <[email protected]> wrote: >>> >>>> // EXPERIMENTAL. DO NOT USE. >>>> // For "map" fields, the name of the field in the enclosed type that >>>> // is the key for this map. For example, suppose we have: >>>> // message Item { >>>> // required string name = 1; >>>> // required string value = 2; >>>> // } >>>> // message Config { >>>> // repeated Item items = 1 [experimental_map_key="name"]; >>>> // } >>>> // In this situation, the map key for Item will be set to "name". >>>> // TODO: Fully-implement this, then remove the "experimental_" prefix. >>>> optional string experimental_map_key = 9; >>>> >>>> Interesting. It basically pins a field within repeated object to be the >>>> key. I like that. That's very aligned with my ideas. This allows other key >>>> types fairly easily. I guess it would make sense to support string, int and >>>> enum by default. I like the syntax I described better though. :) >>>> >>>> On Wed, Oct 6, 2010 at 12:07 PM, David Yu <[email protected]>wrote: >>>> >>>>> In descriptor.proto, you'll see an experimental map field. It's not >>>>> usable atm. >>>>> In the meantime, you could always simulate a map serialization using a >>>>> repeated message with >>>>> odd field numbers as $key and even as $value (sequential). >>>>> >>>>> On Wed, Oct 6, 2010 at 2:23 PM, Igor Gatis <[email protected]>wrote: >>>>> >>>>>> Not sure whether this has been discussed before. In any case... >>>>>> >>>>>> It would be nice to have mapped fields, e.g. key-value pairs. It would >>>>>> work similar to repeated fields, which are implicit maps, e.g 0..N keyed >>>>>> messages. Mapped fields would break from 0..N keys to int or string keys. >>>>>> Integers are very compact and that is very attractive in terms of wire >>>>>> format but settle with integer keys are not really greater than 0..N >>>>>> keys. >>>>>> Thus, string seems more suitable keys of mapped fields. Thus, it seems >>>>>> each >>>>>> item of a mapped field could be defined by the following template-like >>>>>> proto >>>>>> message: >>>>>> >>>>>> message KeyValuePair_of_SomeMessageType { >>>>>> required string key = 1; >>>>>> optional SomeMessageType value = 2; >>>>>> } >>>>>> >>>>>> >>>>>> Let's pick a example. Consider the following messages: >>>>>> >>>>>> message Foo { >>>>>> optional int int_field = 1; >>>>>> ... >>>>>> } >>>>>> >>>>>> message Bar { >>>>>> mapped Foo foo = 1; >>>>>> } >>>>>> >>>>>> Internally, protobuf would read the above code as something like: >>>>>> >>>>>> message Foo { >>>>>> optional int int_field = 1; >>>>>> ... >>>>>> } >>>>>> >>>>>> // Known in code generation time only. >>>>>> message KeyValuePair_of_Foo { >>>>>> required string key = 1; >>>>>> optional Foo value = 2; >>>>>> } >>>>>> >>>>>> message Bar { >>>>>> repeated KeyValuePair_of_Foo foo = 1; >>>>>> } >>>>>> >>>>>> >>>>>> And generated C++ code for Bar would look like: >>>>>> >>>>>> int32 foo_size() const; >>>>>> bool has_foo(const string& key) const; >>>>>> const Foo& foo(const string& key) const; >>>>>> Foo* mutable_foo(const string& key); >>>>>> void put_foo(const string& key, const Foo& foo); >>>>>> void remove_foo(const string& key); >>>>>> const RepeatedPtrField<string>& foo_keys() const; >>>>>> const RepeatedPtrField<const Foo&>& foo_values() const; >>>>>> >>>>>> >>>>>> Thoughts? >>>>>> >>>>>> -Gatis >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Protocol Buffers" group. >>>>>> To post to this group, send email to [email protected]. >>>>>> To unsubscribe from this group, send email to >>>>>> [email protected]<protobuf%[email protected]> >>>>>> . >>>>>> For more options, visit this group at >>>>>> http://groups.google.com/group/protobuf?hl=en. >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> When the cat is away, the mouse is alone. >>>>> - David Yu >>>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Protocol Buffers" group. >>>> To post to this group, send email to [email protected]. >>>> To unsubscribe from this group, send email to >>>> [email protected]<protobuf%[email protected]> >>>> . >>>> For more options, visit this group at >>>> http://groups.google.com/group/protobuf?hl=en. >>>> >>> >>> >> > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
