This sounds great! Communicating using a well-defined JSON spec sounds
like a better idea than trying to communicate using protobufs. One
thing I would suggest is having a protocol version number in there
somewhere so the generator knows what version it's targeting.


On Wed, Oct 28, 2009 at 3:18 PM, Kenton Varda <> wrote:
> Hi all,
> I just had an idea for making protoc more extensible:
> Currently, it is possible to write custom code generators and link against
> libprotoc to build custom protoc-like binaries.  This allows third-party
> implementations to reuse the protoc front-end while keeping development
> independent.  The down side is that third-party implementations must publish
> a different binary rather than being accessible through protoc itself.
>  Also, writing third-party code generators in languages other than C++ is
> tricky.
> It has been proposed before that protoc could use a "plugin" infrastructure
> for code generators.  Third-party generators would be compiled as dynamic
> libraries and loaded by protoc at runtime.  The disadvantage of this
> approach is that dynamic loading works very differently on different
> platforms.  Worse, compiled plugins would be tightly-coupled to a particular
> version of protoc, meaning they would all have to be re-compiled when protoc
> is updated to a new version.
> Instead, I propose a similar architecture, but where each "plugin" is a
> complete *binary*.  protoc would invoke the binary as a sub-process and
> communicate with it over stdin and stdout.  The communication protocol would
> be defined using protocol buffers!
>   message CodeGeneratorRequest {
>     // FileDescriptorProtos for the .proto files listed on the command line
> and everything
>     // they import.
>     optional FileDescriptorSet parsed_files = 1;
>     // The .proto files that were explicitly listed on the command-line.
>     repeated string files_to_generate = 2;
>     // The generator parameter passed on the command-line.
>     optional string parameter = 3;
>     // Directory to which output files should be written.
>     optional string output_directory = 4;
>   }
>   message CodeGeneratorResponse {
>     // Error message.  If non-empty, code generation failed.
>     optional string error = 1;
>     // names of files that were generated.
>     repeated string generated_files = 2;
>   }
> For code generators written in C++, we'd provide a simple front-end library
> for plugins that implements the above protocol in terms of the existing
> CodeGenerator interface.
> Plugins will be allowed to be placed anywhere in PATH and will be identified
> by the file name:
>   protoc-$LANGUAGE
> E.g.:
>   protoc-cpp  (implements --cpp_out)
> Advantages (vs. dynamic libraries):
> * No binary compatibility problem.  New versions of protoc will work fine
> with old plugins and vice versa.  Since the protocol is based on protobufs,
> we can easily extend it without breaking compatibility.  Note that code
> generator binaries will have to link against libprotobuf, but they can
> easily link against a different version than protoc itself links against.
> * Code generators can be written in any language.  To avoid the obvious
> bootstrapping problem when writing a code generator in the language that it
> generates, we could add an option for protoc to communicate with the plugin
> in JSON format instead of protobuf binary format.  In fact, perhaps we
> should always communicate in JSON since it would avoid binary/text
> conversion issues.
> * Code generators can easily define and use custom options that protoc
> itself doesn't know about.
> * If we made the official implementations be plugins as well, then you could
> actually install and use multiple versions of the code generators.  This is
> particularly useful since versions of the runtime library are
> tightly-coupled to versions of protoc, and in some cases you may find
> yourself stuck using an older version of the runtime library.
> * Easier to implement.
> Thoughts?
> >

You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to
To unsubscribe from this group, send email to
For more options, visit this group at

Reply via email to