On Tue, Sep 22, 2009 at 4:35 PM, Exaurdon <alexrichard...@gmail.com> wrote:

> In my particular case I am working with teams from 10+ different
> projects. Each project is generating dozens to hundreds of messages. I
> imagine that trying to get all of those messages into a single message
> would result in a pretty bloated message definition. Since each
> project team uses their own .proto file, (and each team only needs to
> compile and include proto files from other components they communicate
> with) there would not be a central place to define this message,
> though I could use the 'extension' functionality to get around this.

This is pretty much exactly what extensions were designed for.  We had the
exact same problem at Google, and I personally invented something called
"MessageSet" and later extensions (a refinement of MessageSet) to solve the
problem.  You will need a shared proto file with a message definition like:

  message Outer {
    extensions 1000 to max;

Then each project can "extend" it, by writing something like the following
*in their own .proto file*:

  extend Outer {
    optional FooMessage foo_ext = 1234;
    optional BarMessage bar_ext = 1235;

You need to have some system for keeping the extension numbers unique, of

> Another problem would be that I think I would end up needing code (C+
> +) to iterate and call each of the 'has_*' methods. which is less than
> Ideal with a non-trivial set of messages. It looks like I could avoid
> some of this by using the reflection API to get a list of the fields
> that are set.

Yes, you can use the reflection API's ListFields().  But more likely what
will happen is that the eventual consumer of the message will know what type
it is expecting.

> Thanks for the idea, I'll continue to look at it. I still would love
> to be able to iterate through the descriptor pool. It looks like
> internally it is a set of hash_maps, so iteration shouldn't really be
> difficult, (aside from constructing a thread-safe interface I
> suppose?)

Actually, it would be pretty difficult.  The problem is that a
DescriptorPool is typically just a cache in front of a DescriptorDatabase.
 When you look up something that isn't already in the pool's hash_maps, it
falls back to the DescriptorDatabase.  So we'd also need to add a way to
iterate through a database.  But DescriptorDatabases can be huge.  For
instance, we have an implementation inside Google which contains *all*
protobuf types defined across *all* Google projects -- a huge number!  Thus,
DescriptorPool has quite intentionally been designed to avoid iterating over
the contents, because the contents are potentially infinite.

You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to 
For more options, visit this group at 

Reply via email to