[protobuf] Re: Issue 187 in protobuf: Command-line argument to override the "optimize_for" option
Comment #14 on issue 187 by austin.l...@gmail.com: Command-line argument to override the "optimize_for" option https://code.google.com/p/protobuf/issues/detail?id=187 I was able to use Keaton's plugin as a starting point and am using it now. I didn't know that the protobuf compiler was that extensible. Way cool. FYI, it is less lines of code to write the entire plugin in C++ and directly call the C++ generator from there, though it is a bit trickier to get it right. -- You received this message because this project is configured to send all issue notifications to this address. You may adjust your notification preferences at: https://code.google.com/hosting/settings -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at http://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/d/optout.
[protobuf] Re: Issue 187 in protobuf: Command-line argument to override the "optimize_for" option
Comment #12 on issue 187 by ken...@google.com: Command-line argument to override the "optimize_for" option http://code.google.com/p/protobuf/issues/detail?id=187 s/code code/core code/ s/retains/remains/ *sigh* -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: Issue 187 in protobuf: Command-line argument to override the "optimize_for" option
Comment #11 on issue 187 by ken...@google.com: Command-line argument to override the "optimize_for" option http://code.google.com/p/protobuf/issues/detail?id=187 Hi all, Attached is a little protoc plugin written in Python which converts all the inputs to LITE_RUNTIME -- including renaming so that they do not conflict -- and then passes them on to some other plugin. See readme.txt in the archive for details. Basically with this you can do: protoc --litify_out=cpp:. foo.proto And this will produce foo_lite.pb.h and foo_lite.pb.cc, which contain foo's types compiled with optimize_for=LITE_RUNTIME. You can do Java too: protoc --litify_out=java:. foo.proto You can easily modify this plugin's code to produce any arbitrary transformation you want. It's just a Python script that processes the FileDescriptorProtos representing the inputs. I like this approach because it is completely general. There is nothing that you can't express in a Turing-complete language. In contrast, *none* of the other proposals made so far can express even what this simple plugin does (in particular no one has proposed a solution for the renaming part). I also like that this requires no modification whatsoever to protoc. Not just because I want to avoid work, but because keeping the code code as simple as possible ensures that it retains agile. What do you think? Attachments: litify.tar.gz 4.1 KB -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: Issue 187 in protobuf: Command-line argument to override the "optimize_for" option
Comment #10 on issue 187 by esho...@fusionio.com: Command-line argument to override the "optimize_for" option http://code.google.com/p/protobuf/issues/detail?id=187 Admittedly, my suggestion above does not handle imports and that makes it mostly unusable whether done in or outside of protoc. The assumption on my part was that the line inserted would affect all imported files like a C header include directive which simply inserts the text from the included file at that spot. The only thing that makes sense to me and that doesn't get too complicated is that we add a command-line optimization option override that applies to the specified file and all imported files. protoc foo.proto --optimize_for=LITE_RUNTIME I would need to use this to create a lite and a non-lite library so that my applications can choose which to link in without duplicating .proto files. -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: Issue 187 in protobuf: Command-line argument to override the "optimize_for" option
Comment #9 on issue 187 by ken...@google.com: Command-line argument to override the "optimize_for" option http://code.google.com/p/protobuf/issues/detail?id=187 I agree that protoc should provide something. I was merely pointing out that eshober's suggestion can trivially be implemented without protoc's help. Note that the optimization option for gcc and javac is very different from lite vs. non-lite. When you compile a C++ file with -O2, the compiler doesn't need to know what optimization level was used for all the other files you intend to link against -- it doesn't make a difference. But protoc needs to know not just whether the file it is compiling is using LITE_RUNTIME, but also whether all the files imported by that file are using it. This makes the problem significantly more complicated, because if you specified the optimization options for each of those imports on the command line when you compiled them, then you have to somehow express the same information on the command line for the dependency. This could get really awkward: protoc foo.proto --option=foo.proto=optimize_for:LITE_RUNTIME --option=some_other_package/bar.proto=optimize_for:CODE_SIZE --option=yet_another/baz.proto=optimize_for:SPEED To make matters more complicated, imagine this situation: You are linking against a library which includes foo.proto. The library compiles in this proto in lite mode. You, however, want to import the proto into a file that is compiled in regular mode (and you actually need reflection, so you can't just switch to lite). This means you need to compile a copy of foo.proto into your binary, compiled with different options. Because you are linking against a library that already has the lite version compiled in, you need to make sure that your copy doesn't have conflicting symbol names, and so you probably want to alter the package name too. But, you don't want to maintain your own parallel foo.proto with these changes -- you really want to take the one from the other project and apply some command-line options to alter it. What sort of command-line syntax would support this? My concern is that a lot of people think this problem is trivial, but don't realize that the trivial solutions they are thinking of don't actually work, or only work for a smallish subset of cases. I don't want to introduce something that we'll just have to immediately deprecate and replace with something more general. -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: Issue 187 in protobuf: Command-line argument to override the "optimize_for" option
Comment #8 on issue 187 by dima: Command-line argument to override the "optimize_for" option http://code.google.com/p/protobuf/issues/detail?id=187 Now suppose foo.proto is imported by a few other .proto files... You'll have to modify each one of them, too. Every other compiler - gcc, javac, etc. - lets the user specify the optimization level on the command-line. Why not protoc? -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: Issue 187 in protobuf: Command-line argument to override the "optimize_for" option
Comment #7 on issue 187 by ken...@google.com: Command-line argument to override the "optimize_for" option http://code.google.com/p/protobuf/issues/detail?id=187 If you just want to append a string to the file, you can easily do that like: cp foo.proto foo2.proto echo 'option optimize_for LITE_RUNTIME;' >> foo2.proto protoc foo.proto Any reasonable build system should allow you to do this without changing protoc. -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: Issue 187 in protobuf: Command-line argument to override the "optimize_for" option
Comment #6 on issue 187 by esho...@fusionio.com: Command-line argument to override the "optimize_for" option http://code.google.com/p/protobuf/issues/detail?id=187 What about an even more generic implementation where you can optionally poke arbitrary strings into the file from the commandline? So, for example, instead of overriding options like optimize_for we could simply take a string from the command line and prepend it at the top of the file something like this: $ protoc --prepend-string "option optimize_for LITE_RUNTIME;" Obviously, this only helps when there is not already an optimize_for option in the file but this simple feature could handle lots of build issues like optimization. The user would have to be responsible for using it judiciously. Comments? -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: Issue 187 in protobuf: Command-line argument to override the "optimize_for" option
Comment #5 on issue 187 by esho...@fusionio.com: Command-line argument to override the "optimize_for" option http://code.google.com/p/protobuf/issues/detail?id=187 We also need this feature. We're sharing .proto files between applications. One of the apps has a requirement to be optimized for code size and the other apps need to be optimized for speed. I don't care if arbitrary option overrides are supported but if they are it seems to make sense that they only be for file-level options. -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: Issue 187 in protobuf: Command-line argument to override the "optimize_for" option
Comment #4 on issue 187 by ken...@google.com: Command-line argument to override the "optimize_for" option http://code.google.com/p/protobuf/issues/detail?id=187 I agree, we should be able to override options on the command-line. The only problem is that it's unclear how far this support needs to go. Should you be able to override message-level and field-level options, or just file-level? Should an override apply to an individual file, or should it apply to all the files it imports, too? In the case of optimize_for, we'd probably want the override to apply to imports, but for something like java_package we probably don't. -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: Issue 187 in protobuf: Command-line argument to override the "optimize_for" option
Comment #3 on issue 187 by mike.weller: Command-line argument to override the "optimize_for" option http://code.google.com/p/protobuf/issues/detail?id=187 This is something I've always thought was needed. It makes much more sense to me to define these kinds of options on the command line so you can easily switch them out as part of a build process. This is especially useful when you are targeting multiple languages, as mentioned in the original post. -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: Issue 187 in protobuf: Command-line argument to override the "optimize_for" option
Comment #2 on issue 187 by n...@chromium.org: Command-line argument to override the "optimize_for" option http://code.google.com/p/protobuf/issues/detail?id=187 Would it be much more difficult to support arbitrary option overrides on the command line? -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: Issue 187 in protobuf: Command-line argument to override the "optimize_for" option
Comment #1 on issue 187 by dima: Command-line argument to override the "optimize_for" option http://code.google.com/p/protobuf/issues/detail?id=187 Actually, the patch is wrong... It overrides optimize_for if it's set in .proto files, but does not override the default value. -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
