Hi all,
I just had an idea for making protoc more extensible:
Currently, it is possible to write custom code generators and link against
libprotoc to build custom protoc-like binaries. This allows third-party
implementations to reuse the protoc front-end while keeping development
independent. The down side is that third-party implementations must publish
a different binary rather than being accessible through protoc itself.
Also, writing third-party code generators in languages other than C++ is
tricky.
It has been proposed before that protoc could use a "plugin" infrastructure
for code generators. Third-party generators would be compiled as dynamic
libraries and loaded by protoc at runtime. The disadvantage of this
approach is that dynamic loading works very differently on different
platforms. Worse, compiled plugins would be tightly-coupled to a particular
version of protoc, meaning they would all have to be re-compiled when protoc
is updated to a new version.
Instead, I propose a similar architecture, but where each "plugin" is a
complete *binary*. protoc would invoke the binary as a sub-process and
communicate with it over stdin and stdout. The communication protocol would
be defined using protocol buffers!
message CodeGeneratorRequest {
// FileDescriptorProtos for the .proto files listed on the command line
and everything
// they import.
optional FileDescriptorSet parsed_files = 1;
// The .proto files that were explicitly listed on the command-line.
repeated string files_to_generate = 2;
// The generator parameter passed on the command-line.
optional string parameter = 3;
// Directory to which output files should be written.
optional string output_directory = 4;
}
message CodeGeneratorResponse {
// Error message. If non-empty, code generation failed.
optional string error = 1;
// names of files that were generated.
repeated string generated_files = 2;
}
For code generators written in C++, we'd provide a simple front-end library
for plugins that implements the above protocol in terms of the existing
CodeGenerator interface.
Plugins will be allowed to be placed anywhere in PATH and will be identified
by the file name:
protoc-$LANGUAGE
E.g.:
protoc-cpp (implements --cpp_out)
Advantages (vs. dynamic libraries):
* No binary compatibility problem. New versions of protoc will work fine
with old plugins and vice versa. Since the protocol is based on protobufs,
we can easily extend it without breaking compatibility. Note that code
generator binaries will have to link against libprotobuf, but they can
easily link against a different version than protoc itself links against.
* Code generators can be written in any language. To avoid the obvious
bootstrapping problem when writing a code generator in the language that it
generates, we could add an option for protoc to communicate with the plugin
in JSON format instead of protobuf binary format. In fact, perhaps we
should always communicate in JSON since it would avoid binary/text
conversion issues.
* Code generators can easily define and use custom options that protoc
itself doesn't know about.
* If we made the official implementations be plugins as well, then you could
actually install and use multiple versions of the code generators. This is
particularly useful since versions of the runtime library are
tightly-coupled to versions of protoc, and in some cases you may find
yourself stuck using an older version of the runtime library.
* Easier to implement.
Thoughts?
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~----------~----~----~----~------~----~------~--~---
