Re: [protobuf] Python installation does not build plugin_pb2
I guess it would not be a big deal to include this code, since Python won't load it until it is explicitly imported anyway. Can someone file a bug to have plugin_pb2.py included in the base installation for the next release? On Sat, Oct 9, 2010 at 11:50 AM, Gregory Szorc wrote: > I agree with Louis-Marie: I would like the compiler file(s) included in > the Python package. > > Every protoc plugin written in Python will need to import the > google.protobuf.compiler.plugin_pb2 module. This means that any user > wishing to use that plugin (it isn't limited to plugin developers > themselves) will be out of luck with a default Python protobuf install, > which omits this file. > > There are a couple of workarounds (non-exhaustive list): > > 1) compiler plugin packages can copy generated code from plugin_pb2.py to > their project/module and distribute it themselves. > 2) Python plugins can tell end-users to manually compile and > install plugin_pb2.py. > 3) Don't use Python for the compiler plugin. > > So, these workarounds require me to feel dirty that I'm copying code > instead of using a library verbatim (1), take a risk that the produced > plugin_pb2.py file won't change with a future release (1), force all users > of my plugin to jump through extra steps (2), and/or write my compiler > plugin in a language in which I'm not as proficient (3). > > While it is true that the majority of installations will likely never > utilize the compiler plugin module, for those that do, the barrier to > entry can be increased and resolving the issue can be extremely frustrating, > especially if compiler plugins don't have the documentation telling users > how to resolve the import failure. Since it doesn't "just work," the > experience tarnishes the image of the plugin and of protocol buffers itself. > > The only argument against inclusion so far is package bloat. On my > system, plugin_pb2.py and plugin_pb2.pyc are 6247 and 4750 bytes, > respectively. The current uncompressed egg weighs in around 500K. Are we > really objecting over a 11k/2% size increase? That seems trivial to me, > especially since the added file(s) won't conflict with any existing ones and > thus there is little risk to adding it. > > While I would prefer that the Python installer be updated to include > compiler.plugin_pb2, at the least could the documentation be updated to > reflect a preferred workaround? (And I don't think forcing the use of a > program packager like PyInstaller is the correct route either.) > > Greg > > *From:* Jason Hsueh > *Sent:* Thursday, October 07, 2010 1:55 PM > *To:* Louis-Marie > *Cc:* Kenton Varda ; protobuf@googlegroups.com > *Subject:* Re: [protobuf] Python installation does not build plugin_pb2 > > As Kenton said, including plugin.proto would bloat the core library. Only > people implementing proto compilers such as yourself need to use it. > > On the C++ side, you would typically just build a statically linked binary > that has all of the plugin generated code linked in. It's not included in > the C++ core library either. There are various mechanisms to package python > programs, e.g. http://www.pyinstaller.org/, that would allow you to give > users a functioning plugin program without requiring that they install the > generated modules for plugin.proto. > > > On Wed, Oct 6, 2010 at 12:42 AM, Louis-Marie wrote: > >> Thanks for your answer. >> >> To give you a little bit more information, here's what I'm trying to >> do. I want to deliver a tool using a custom protoc plugin implemented >> in python, so that end user can generate code from its own proto >> files. >> >> There would be nothing special to do before using this plugin, but >> since it depends on plugin_pb2, I need to find the plugin.proto file >> (using pkg-config), compile it, and put it in some appropriate >> location. >> >> Also, I can't put it in its "natural" parent python package >> (google.protobuf) which is already provided by protobuf installation. >> On the other side, the c++ code is generated (and I guess, compiled >> into the shared library), which makes it a little bit more >> straightforward to write c++ plugins. >> >> What would be the best way to achieve what I'm trying to do? If the >> python library size increase is not acceptable there, couldn't the >> plugin_pb2 file be generated in an independent location, so that one >> could still rely on it being present on any protobuf install? >> >> Thanks for your advices, >> >> Louis-Marie >> >> >> 2010/10/6 Kenton Vard
Re: [protobuf] Python installation does not build plugin_pb2
I agree with Louis-Marie: I would like the compiler file(s) included in the
Python package.
Every protoc plugin written in Python will need to import the
google.protobuf.compiler.plugin_pb2 module. This means that any user wishing to
use that plugin (it isn't limited to plugin developers themselves) will be out
of luck with a default Python protobuf install, which omits this file.
There are a couple of workarounds (non-exhaustive list):
1) compiler plugin packages can copy generated code from plugin_pb2.py to their
project/module and distribute it themselves.
2) Python plugins can tell end-users to manually compile and install
plugin_pb2.py.
3) Don't use Python for the compiler plugin.
So, these workarounds require me to feel dirty that I'm copying code instead of
using a library verbatim (1), take a risk that the produced plugin_pb2.py file
won't change with a future release (1), force all users of my plugin to jump
through extra steps (2), and/or write my compiler plugin in a language in which
I'm not as proficient (3).
While it is true that the majority of installations will likely never utilize
the compiler plugin module, for those that do, the barrier to entry can be
increased and resolving the issue can be extremely frustrating, especially if
compiler plugins don't have the documentation telling users how to resolve the
import failure. Since it doesn't "just work," the experience tarnishes the
image of the plugin and of protocol buffers itself.
The only argument against inclusion so far is package bloat. On my system,
plugin_pb2.py and plugin_pb2.pyc are 6247 and 4750 bytes, respectively. The
current uncompressed egg weighs in around 500K. Are we really objecting over a
11k/2% size increase? That seems trivial to me, especially since the added
file(s) won't conflict with any existing ones and thus there is little risk to
adding it.
While I would prefer that the Python installer be updated to include
compiler.plugin_pb2, at the least could the documentation be updated to reflect
a preferred workaround? (And I don't think forcing the use of a program
packager like PyInstaller is the correct route either.)
Greg
From: Jason Hsueh
Sent: Thursday, October 07, 2010 1:55 PM
To: Louis-Marie
Cc: Kenton Varda ; protobuf@googlegroups.com
Subject: Re: [protobuf] Python installation does not build plugin_pb2
As Kenton said, including plugin.proto would bloat the core library. Only
people implementing proto compilers such as yourself need to use it.
On the C++ side, you would typically just build a statically linked binary that
has all of the plugin generated code linked in. It's not included in the C++
core library either. There are various mechanisms to package python programs,
e.g. http://www.pyinstaller.org/, that would allow you to give users a
functioning plugin program without requiring that they install the generated
modules for plugin.proto.
On Wed, Oct 6, 2010 at 12:42 AM, Louis-Marie wrote:
Thanks for your answer.
To give you a little bit more information, here's what I'm trying to
do. I want to deliver a tool using a custom protoc plugin implemented
in python, so that end user can generate code from its own proto
files.
There would be nothing special to do before using this plugin, but
since it depends on plugin_pb2, I need to find the plugin.proto file
(using pkg-config), compile it, and put it in some appropriate
location.
Also, I can't put it in its "natural" parent python package
(google.protobuf) which is already provided by protobuf installation.
On the other side, the c++ code is generated (and I guess, compiled
into the shared library), which makes it a little bit more
straightforward to write c++ plugins.
What would be the best way to achieve what I'm trying to do? If the
python library size increase is not acceptable there, couldn't the
plugin_pb2 file be generated in an independent location, so that one
could still rely on it being present on any protobuf install?
Thanks for your advices,
Louis-Marie
2010/10/6 Kenton Varda :
> It's not generated because none of the python implementation actually uses
> it. So, generating it and including it in the egg would just increase the
> library size for everyone, when most people don't need it.
> What makes you feel uncomfortable about generating it yourself?
>
> On Fri, Oct 1, 2010 at 6:01 AM, Louis-Marie wrote:
>>
>> Hi all,
>>
>> It looks like python installation of protocol buffers does not
>> generate the google.protobuf.compiler.plugin_pb2 python file, while
>> google.protobuf.descriptor_pb2 is explicitly generated by
>> protobuf/python/setup.py
>>
>> generate_proto("../src/google/protobuf/descriptor.proto")
>>
&
Re: [protobuf] Python installation does not build plugin_pb2
As Kenton said, including plugin.proto would bloat the core library. Only
people implementing proto compilers such as yourself need to use it.
On the C++ side, you would typically just build a statically linked binary
that has all of the plugin generated code linked in. It's not included in
the C++ core library either. There are various mechanisms to package python
programs, e.g. http://www.pyinstaller.org/, that would allow you to give
users a functioning plugin program without requiring that they install the
generated modules for plugin.proto.
On Wed, Oct 6, 2010 at 12:42 AM, Louis-Marie wrote:
> Thanks for your answer.
>
> To give you a little bit more information, here's what I'm trying to
> do. I want to deliver a tool using a custom protoc plugin implemented
> in python, so that end user can generate code from its own proto
> files.
>
> There would be nothing special to do before using this plugin, but
> since it depends on plugin_pb2, I need to find the plugin.proto file
> (using pkg-config), compile it, and put it in some appropriate
> location.
>
> Also, I can't put it in its "natural" parent python package
> (google.protobuf) which is already provided by protobuf installation.
> On the other side, the c++ code is generated (and I guess, compiled
> into the shared library), which makes it a little bit more
> straightforward to write c++ plugins.
>
> What would be the best way to achieve what I'm trying to do? If the
> python library size increase is not acceptable there, couldn't the
> plugin_pb2 file be generated in an independent location, so that one
> could still rely on it being present on any protobuf install?
>
> Thanks for your advices,
>
> Louis-Marie
>
>
> 2010/10/6 Kenton Varda :
> > It's not generated because none of the python implementation actually
> uses
> > it. So, generating it and including it in the egg would just increase
> the
> > library size for everyone, when most people don't need it.
> > What makes you feel uncomfortable about generating it yourself?
> >
> > On Fri, Oct 1, 2010 at 6:01 AM, Louis-Marie wrote:
> >>
> >> Hi all,
> >>
> >> It looks like python installation of protocol buffers does not
> >> generate the google.protobuf.compiler.plugin_pb2 python file, while
> >> google.protobuf.descriptor_pb2 is explicitly generated by
> >> protobuf/python/setup.py
> >>
> >> generate_proto("../src/google/protobuf/descriptor.proto")
> >>
> >> Shouldn't the plugin.proto file be compiled and installed the same
> >> way? Maybe I am missing something there, be I feel very uncomfortable
> >> recompiling it when I need to write a plugin.
> >>
> >> Thanks,
> >>
> >> Louis-Marie
> >>
> >> --
> >> You received this message because you are subscribed to the Google
> Groups
> >> "Protocol Buffers" group.
> >> To post to this group, send email to proto...@googlegroups.com.
> >> To unsubscribe from this group, send email to
> >> protobuf+unsubscr...@googlegroups.com
> .
> >> For more options, visit this group at
> >> http://groups.google.com/group/protobuf?hl=en.
> >>
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Protocol Buffers" group.
> > To post to this group, send email to proto...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > protobuf+unsubscr...@googlegroups.com
> .
> > For more options, visit this group at
> > http://groups.google.com/group/protobuf?hl=en.
> >
>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To post to this group, send email to proto...@googlegroups.com.
> To unsubscribe from this group, send email to
> protobuf+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/protobuf?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Python installation does not build plugin_pb2
Thanks for your answer.
To give you a little bit more information, here's what I'm trying to
do. I want to deliver a tool using a custom protoc plugin implemented
in python, so that end user can generate code from its own proto
files.
There would be nothing special to do before using this plugin, but
since it depends on plugin_pb2, I need to find the plugin.proto file
(using pkg-config), compile it, and put it in some appropriate
location.
Also, I can't put it in its "natural" parent python package
(google.protobuf) which is already provided by protobuf installation.
On the other side, the c++ code is generated (and I guess, compiled
into the shared library), which makes it a little bit more
straightforward to write c++ plugins.
What would be the best way to achieve what I'm trying to do? If the
python library size increase is not acceptable there, couldn't the
plugin_pb2 file be generated in an independent location, so that one
could still rely on it being present on any protobuf install?
Thanks for your advices,
Louis-Marie
2010/10/6 Kenton Varda :
> It's not generated because none of the python implementation actually uses
> it. So, generating it and including it in the egg would just increase the
> library size for everyone, when most people don't need it.
> What makes you feel uncomfortable about generating it yourself?
>
> On Fri, Oct 1, 2010 at 6:01 AM, Louis-Marie wrote:
>>
>> Hi all,
>>
>> It looks like python installation of protocol buffers does not
>> generate the google.protobuf.compiler.plugin_pb2 python file, while
>> google.protobuf.descriptor_pb2 is explicitly generated by
>> protobuf/python/setup.py
>>
>> generate_proto("../src/google/protobuf/descriptor.proto")
>>
>> Shouldn't the plugin.proto file be compiled and installed the same
>> way? Maybe I am missing something there, be I feel very uncomfortable
>> recompiling it when I need to write a plugin.
>>
>> Thanks,
>>
>> Louis-Marie
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Protocol Buffers" group.
>> To post to this group, send email to proto...@googlegroups.com.
>> To unsubscribe from this group, send email to
>> protobuf+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/protobuf?hl=en.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To post to this group, send email to proto...@googlegroups.com.
> To unsubscribe from this group, send email to
> protobuf+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/protobuf?hl=en.
>
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Python installation does not build plugin_pb2
It's not generated because none of the python implementation actually uses
it. So, generating it and including it in the egg would just increase the
library size for everyone, when most people don't need it.
What makes you feel uncomfortable about generating it yourself?
On Fri, Oct 1, 2010 at 6:01 AM, Louis-Marie wrote:
> Hi all,
>
> It looks like python installation of protocol buffers does not
> generate the google.protobuf.compiler.plugin_pb2 python file, while
> google.protobuf.descriptor_pb2 is explicitly generated by
> protobuf/python/setup.py
>
> generate_proto("../src/google/protobuf/descriptor.proto")
>
> Shouldn't the plugin.proto file be compiled and installed the same
> way? Maybe I am missing something there, be I feel very uncomfortable
> recompiling it when I need to write a plugin.
>
> Thanks,
>
> Louis-Marie
>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To post to this group, send email to proto...@googlegroups.com.
> To unsubscribe from this group, send email to
> protobuf+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/protobuf?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
