On Wed, Jan 31, 2018 at 2:57 PM, Jonnny <[email protected]> wrote: > By 'Protobuf Python Library' you mean the Python Package called `protobuf > ? I think It's already included in my package as dependency. But Google > protobuf is just one example. Most people around here using Scala, and they > all have 'scalapb' imported into their proto files. The path is `*scalapb* > /*scalapb*.*proto`. *It's not something I can chance as it's a third > party proto, and it's not a Google protobuf so using the `protobuf` package > won't solve this issue. I can manually add it into my `/usr/local/include` > folder, but this mean the package is not self-contained and the user needs > to compile his own scalapb pb2 file, and manually add it to > `/usr/local/include` if he wants my package to work. > Is there not a Pypi package containing scalapb/scalapb_pb2.py already? How about ask the author of scalapb/scalapb.proto to provide such a python package?
> > It means again, I have to keep the `scalapb` folder set into the root of > my project, or play around with PYTHONPATH. I wasn't able to find online > Python projects/packages that uses Python complied proto code, so I was > thinking perhaps there is a way around it. But if there is not a better > way, and that's consider the 'standard', I'm guessing I can do one of the > three below: > > 1. On a normal project, `scalapb` should be sitting on the root of the > project (ie, the folder where I'm running the script from). > 2. Packages are more complex, because the 'root' includes things like > `setup.py` and `README.srt`, but not the actual package code. The package > sits in a directory inside the root Folder. For example in the Request > package (https://github.com/requests/requests), the actual code run from > within the `request` folder. So even if I'll keep the generated code > directory structure in the `requests` folder, it won't work necause the > relative path starts from the Root folder and not `requests` folder. Adding > the folder structure into the root of the project is messy (because you > need to defines those as package in `setup.py`, and perhaps even supply > __init__.py file). > 3. Playing with PYTHONPATH like so: > > > import sys > sys.path.append('./my_package_name/proto') > > > This code need to be called every-time the package is called (probably need > to sit inside the `__init__.py I'm guessing). > > > There is a 4th option, but one that I do not like much. In case I have > small amount of `pb2` files, I can manually edit the `pb2` imports by > adding my folder after the code is compiled. > > I would just go ahead with (1). What's so bad about having a scalapb directory in your root directory? If you are concerned with conflicts with other project that may also include scalapb, you can just create a separate python package for the scalapb/scalapb_pb2.py, deploy it to pypi, and share it with all other projects that depend on the same thing. > > > > > > On Wednesday, January 31, 2018 at 11:39:08 PM UTC+2, Feng Xiao wrote: >> >> >> >> On Wed, Jan 31, 2018 at 8:54 AM, Jonnny <[email protected]> wrote: >> >>> Hello everyone, >>> >>> I have a couple of micro-services, and each has couple of proto files. >>> It's common for one micro-service proto file to reference another service >>> proto file using import like so: >>> >>> import "google/protobuf/struct.proto"; >>> import "myService/v1/message.proto"; >>> import "myOtherService/v2/parser.proto"; >>> >>> We use a Gradle script that's responsible on creating the folder >>> structure and fetch all proto dependencies from different git repositories. >>> So in the example above the Gradle script will output will be a folder >>> named `google/protobuff`, `myServicev1/v1` and `myOtherService/v2` - each >>> of the folders will hold the proto files needed. Once the folder structure >>> is OK, I can use `protoc` to generate the Python code. >>> >>> Here's where the problem begins. The generated `_pb2` code import >>> modules based on the folder structure. So the import inside the `_pb2` file >>> will look like so: >>> >>> from google.protobuf import struct_pb2 as >>> google_dot_protobuf_dot_struct__pb2 >>> >>> >>> That's a relative path import, and it means that in order for my Python >>> program to work, the root folder where the Python script running from, >>> should have the `google/protobuff` folder for this import to work.Now >>> because I have multiple micro-services, I don't appreciated the fact I have >>> to add 3 new folders in my main Python project (it's even more painful when >>> you create a Pip package). What I was hoping to do is to open a directory >>> called `proto`, and add all that directory structure into it - but because >>> imports are relative, this won't work. I tried to to play with couple of >>> `protoc` options like `PROTO_PATH`, or run `protoc` from a parent >>> directory, so the `_pb2` files will get created with a different relative >>> path I can work with like so: >>> >>> from proto.google.protobuf import struct_pb2 as >>> google_dot_protobuf_dot_struct__pb2 >>> >>> But I wasn't able to do so. A dirty option would be adding the `proto` >>> folder into my `PYTHONPATH`. It's not pretty, and I'm guessing there is >>> probably a simple solution for that I'm missing with `protoc`. >>> >> You can probably change the path for your own message.proto/parser.proto, >> but for protobuf's google/protobuf/struct.proto (and any other protos under >> google/protobuf), it's the only option to import it as >> "google/protobuf/struct.proto". Nothing else would work. It just has to >> be imported as "google/protobuf/struct.proto" in any .proto file and has to >> be imported as "google.protobuf.struct_pb2" in any python file. It's part >> of protobuf public API and won't (and can't) be changed. >> >> Where is protobuf python library installed on your machine? If you have >> it installed, google.protobuf.struct_pb2 should be part of it and you >> shouldn't need to put google/protobuf/struct_pb2.py in your own project >> directory. >> >> >>> >>> Thanks! >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Protocol Buffers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> Visit this group at https://groups.google.com/group/protobuf. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > You received this message because you are subscribed to the Google Groups > "Protocol Buffers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/protobuf. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/d/optout.
