Re: [protobuf] Re: Protocol Buffers 2.6.0 or 2.6.0 Unable to build on AIX 5.3x6 xlC compiler using gmake

['Feng Xiao' via Protocol Buffers]

On Tue, Jul 12, 2016 at 3:20 PM, Alban Lefebvre 
wrote:

> Having the same issue here.
> Will version 2.6 be supporting AIX platform?
>
We can't fix the problem in the already released versions. Could you try if
the problem is fixed in the newer 3.0.0-beta-3 version?


>
>
> On Wednesday, July 22, 2015 at 5:17:01 PM UTC-4, Dannis Yang wrote:
>>
>> Per install instruction, I run ./configure CXX=xlC successfully but
>> immediate gmake failed as following:
>>
>> bash-2.05a# gmake
>> gmake  all-recursive
>> gmake[1]: Entering directory `/home/dyang/protobuf/protobuf-2.6.1'
>> Making all in .
>> gmake[2]: Entering directory `/home/dyang/protobuf/protobuf-2.6.1'
>> gmake[2]: Leaving directory `/home/dyang/protobuf/protobuf-2.6.1'
>> Making all in src
>> gmake[2]: Entering directory `/home/dyang/protobuf/protobuf-2.6.1/src'
>> source='google/protobuf/stubs/atomicops_internals_x86_gcc.cc'
>> object='google/protobuf/stubs/atomicops_internals_x86_gcc.
>> lo' libtool=yes \
>> DEPDIR=.deps depmode=xlc /bin/sh ../depcomp \
>> /bin/sh ../libtool  --tag=CXX   --mode=compile xlC -DHAVE_CONFIG_H -I.
>> -I..-D_THREAD_SAFE   -DNDEBUG -c -o google/pr
>> otobuf/stubs/atomicops_internals_x86_gcc.lo
>> google/protobuf/stubs/atomicops_internals_x86_gcc.cc
>> libtool: compile:  xlC -DHAVE_CONFIG_H -I. -I.. -D_THREAD_SAFE -DNDEBUG
>> -c google/protobuf/stubs/atomicops_internals_x86
>> _gcc.cc
>> -Wp,-qmakedep=gcc,-MFgoogle/protobuf/stubs/.deps/atomicops_internals_x86_gcc.TPlo
>>  -DPIC -o google/protobuf/stub
>> s/.libs/atomicops_internals_x86_gcc.o
>> "./google/protobuf/stubs/atomicops.h", line 214.1: 1540-0825 (W) The
>> character "#" is not allowed.
>> "./google/protobuf/stubs/platform_macros.h", line 90.1: 1540-0825 (W) The
>> character "#" is not allowed.
>> "./google/protobuf/stubs/platform_macros.h", line 90.1: 1540-0040 (S) The
>> text ""Host platform was not detected as suppo
>> rted by protobuf"" is unexpected.  "error" may be undeclared or ambiguous.
>> gmake[2]: *** [google/protobuf/stubs/atomicops_internals_x86_gcc.lo]
>> Error 1
>> gmake[2]: Leaving directory `/home/dyang/protobuf/protobuf-2.6.1/src'
>> gmake[1]: *** [all-recursive] Error 1
>> gmake[1]: Leaving directory `/home/dyang/protobuf/protobuf-2.6.1'
>> gmake: *** [all] Error 2
>>
>> Has anyone successfully built Protocol Buffers on AIX?
>>
>> Thank you
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to protobuf+unsubscr...@googlegroups.com.
> To post to this group, send email to protobuf@googlegroups.com.
> Visit this group at https://groups.google.com/group/protobuf.
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[protobuf] Re: Protocol Buffers 2.6.0 or 2.6.0 Unable to build on AIX 5.3x6 xlC compiler using gmake

[Alban Lefebvre]

Having the same issue here.
Will version 2.6 be supporting AIX platform?

On Wednesday, July 22, 2015 at 5:17:01 PM UTC-4, Dannis Yang wrote:
>
> Per install instruction, I run ./configure CXX=xlC successfully but 
> immediate gmake failed as following:
>
> bash-2.05a# gmake
> gmake  all-recursive
> gmake[1]: Entering directory `/home/dyang/protobuf/protobuf-2.6.1'
> Making all in .
> gmake[2]: Entering directory `/home/dyang/protobuf/protobuf-2.6.1'
> gmake[2]: Leaving directory `/home/dyang/protobuf/protobuf-2.6.1'
> Making all in src
> gmake[2]: Entering directory `/home/dyang/protobuf/protobuf-2.6.1/src'
> source='google/protobuf/stubs/atomicops_internals_x86_gcc.cc' 
> object='google/protobuf/stubs/atomicops_internals_x86_gcc.
> lo' libtool=yes \
> DEPDIR=.deps depmode=xlc /bin/sh ../depcomp \
> /bin/sh ../libtool  --tag=CXX   --mode=compile xlC -DHAVE_CONFIG_H -I. 
> -I..-D_THREAD_SAFE   -DNDEBUG -c -o google/pr
> otobuf/stubs/atomicops_internals_x86_gcc.lo 
> google/protobuf/stubs/atomicops_internals_x86_gcc.cc
> libtool: compile:  xlC -DHAVE_CONFIG_H -I. -I.. -D_THREAD_SAFE -DNDEBUG -c 
> google/protobuf/stubs/atomicops_internals_x86
> _gcc.cc 
> -Wp,-qmakedep=gcc,-MFgoogle/protobuf/stubs/.deps/atomicops_internals_x86_gcc.TPlo
>  
>  -DPIC -o google/protobuf/stub
> s/.libs/atomicops_internals_x86_gcc.o
> "./google/protobuf/stubs/atomicops.h", line 214.1: 1540-0825 (W) The 
> character "#" is not allowed.
> "./google/protobuf/stubs/platform_macros.h", line 90.1: 1540-0825 (W) The 
> character "#" is not allowed.
> "./google/protobuf/stubs/platform_macros.h", line 90.1: 1540-0040 (S) The 
> text ""Host platform was not detected as suppo
> rted by protobuf"" is unexpected.  "error" may be undeclared or ambiguous.
> gmake[2]: *** [google/protobuf/stubs/atomicops_internals_x86_gcc.lo] Error 
> 1
> gmake[2]: Leaving directory `/home/dyang/protobuf/protobuf-2.6.1/src'
> gmake[1]: *** [all-recursive] Error 1
> gmake[1]: Leaving directory `/home/dyang/protobuf/protobuf-2.6.1'
> gmake: *** [all] Error 2
>
> Has anyone successfully built Protocol Buffers on AIX? 
>
> Thank you
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[protobuf] java.lang.NoClassDefFoundError: com/google/protobuf/MessageOrBuilder

[mindentropy]

Hi,

I am using protobuf 2.6.1 ( 
https://developers.google.com/protocol-buffers/docs/downloads ). I compiled 
the code and used maven to build the protobuf-java-2.6.1.jar

In the addressbook example I removed the initial exit scenario. Whenever 
the code hits the first protobuf function call I get a 
java.lang.NoClassDefFoundError: com/google/protobuf/MessageOrBuilder

When I compile I do a javac ProtoTesting.java 
/com/somedir/trial/TestProtos.java -classpath 
/protobuf-java-2.6.1.jar
When I run I do a java ProtoTesting. After this I get the above error.

Please let me know what I am doing wrong.

Thanks.



-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] Question about Go protobufs and import_prefix

['Ross Light' via Protocol Buffers]

Zellyn and I talked a little bit at Gophercon about this. I think some kind
of wrapper is in order for his use case: it may actually be a filter
between protoc and protoc-gen-go.

On Tue, Jul 12, 2016, 8:49 AM Damien Neil  wrote:

> If I'm following this correctly, your core problem is that protoc doesn't
> understand vendored paths as used by the go tool. For example:
>
> - You have a proto file located in src/square/up/protos/square.proto.
> - square.proto imports "
> github.com/golang/protobuf/ptypes/timestamp/timestamp.proto".
> - timestamp.proto is actually located in "src/square/up/vendor/
> github.com/.../timestamp.proto".
>
> You can't run "protoc --go_out=. square/up/protos/square.proto", because
> protoc won't be able to locate the vendored copy of timestamp.proto. If it
> did, everything would work correctly without setting an import_prefix.
>
> It would be nice for Go users if protoc did understand vendored paths.
> Failing that, however, I think you can get what you want with a very small
> wrapper around protoc. The wrapper can copy a proto and all its
> dependencies into a temp directory (pulling dependencies from vendored
> paths as necessary), run protoc there, and copy the result back. e.g., in
> the case of the above example:
>
> $ mkdir -p /tmp/x/square/up/protos
> $ cp src/square/up/protos/square.proto /tmp/x/square/up/protos
> $ mkdir -p /tmp/x/github.com/golang/protobuf/ptypes/timestamp
> $ cp square/up/vendor/
> github.com/golang/protobuf/ptypes/timestamp/timestamp.proto
> github.com/golang/protobuf/ptypes/timestamp
> $ cd /tmp/x && protoc -go_out=. square/up/protos/square.proto
> $ cp /tmp/x/square/up/protos/square.pb.go square/up/protos
>
> The generated .go files will reference non-vendored paths, which the Go
> compiler will resolve to the correct vendored directory.
>
> Does that seem like it would work for you?
>
> On Sun, Jul 10, 2016 at 12:32 PM, Zellyn Hunter  wrote:
>
>> Usually not too tricky. The problem is that protos and most other
>> programming languages prohibit circular imports at the file level, and Go
>> does it at the package level. The fact that protoc and the other languages
>> are okay with a topology ensures there is a valid Go repackaging that will
>> work.
>>
>> Zellyn
>>
>> On Fri, Jul 8, 2016, 6:52 PM 'Josh Haberman' via Protocol Buffers <
>> protobuf@googlegroups.com> wrote:
>>
>>> I don't know the background of the Go import system or go_package
>>> option. However this statement concerns me a little:
>>>
>>> > We have to use go_package to reorganize things that are fine in
>>> Java/Ruby, but would be circular package imports in Go.
>>>
>>> Is this implying that certain .proto files would generate invalid Go
>>> code until you manually insert some go_package statements? How tricky is it
>>> to do this manual untangling?
>>>
>>>
>>> On Friday, July 1, 2016 at 7:53:03 PM UTC-7, Zellyn wrote:

 Oh wow. Thanks for thinking about it so carefully!

 I should mention: before you could actually specify full package paths
 to the Go proto plugin, earlier versions of our wrapper (
 https://github.com/square/goprotowrap) just forcibly manhandled things
 into the shape we wanted by explicitly specifying -M parameter import
 rewrites for *every single* import, and renaming the output files into
 place.

 So it's definitely possible to shape things as you want. However, I
 would prefer to find a solution that works with the official plugin,
 because I don't believe our setup is remarkable or unusual: as GRPC sees an
 increase in use as a cross-language RPC mechanism, everyone else is going
 to struggle with the same things.

 Zellyn

 On Fri, Jul 1, 2016 at 7:32 PM Ross Light  wrote:

> Sorry for the delayed response!  I've written about 3-4 draft replies
> over this week that I've discarded because they don't meet your
> requirements.  I'll think about it more over the long weekend and get back
> to you.
>
> Cheers,
> -Ross
>
>
> On Fri, Jul 1, 2016 at 3:11 PM Jie Luo  wrote:
>
>> +Ross who is working on Go protobuf.
>>
>> Ross, do you have any comments or suggestions?
>>
>> On Fri, Jun 24, 2016 at 8:35 AM, Zellyn  wrote:
>>
>>> Hi folks,
>>>
>>> Apologies in advance for the complex email. It takes a bit of
>>> explaining to set up what we're having trouble with.
>>>
>>> I would like to lay out how we generate protos in Go (using our
>>> unfortunately forked version of the protoc plugin), and ask for 
>>> suggestions
>>> as to how to accomplish the same thing with the unforked version. In the
>>> process, I wish to ask questions about the post-vendor-experiment 
>>> utility
>>> of the current behavior of import_prefix, hopefully generating more
>>> discussion than I managed to 

Re: [protobuf] Question about Go protobufs and import_prefix

['Damien Neil' via Protocol Buffers]

If I'm following this correctly, your core problem is that protoc doesn't
understand vendored paths as used by the go tool. For example:

- You have a proto file located in src/square/up/protos/square.proto.
- square.proto imports "
github.com/golang/protobuf/ptypes/timestamp/timestamp.proto".
- timestamp.proto is actually located in "src/square/up/vendor/
github.com/.../timestamp.proto".

You can't run "protoc --go_out=. square/up/protos/square.proto", because
protoc won't be able to locate the vendored copy of timestamp.proto. If it
did, everything would work correctly without setting an import_prefix.

It would be nice for Go users if protoc did understand vendored paths.
Failing that, however, I think you can get what you want with a very small
wrapper around protoc. The wrapper can copy a proto and all its
dependencies into a temp directory (pulling dependencies from vendored
paths as necessary), run protoc there, and copy the result back. e.g., in
the case of the above example:

$ mkdir -p /tmp/x/square/up/protos
$ cp src/square/up/protos/square.proto /tmp/x/square/up/protos
$ mkdir -p /tmp/x/github.com/golang/protobuf/ptypes/timestamp
$ cp square/up/vendor/
github.com/golang/protobuf/ptypes/timestamp/timestamp.proto
github.com/golang/protobuf/ptypes/timestamp
$ cd /tmp/x && protoc -go_out=. square/up/protos/square.proto
$ cp /tmp/x/square/up/protos/square.pb.go square/up/protos

The generated .go files will reference non-vendored paths, which the Go
compiler will resolve to the correct vendored directory.

Does that seem like it would work for you?

On Sun, Jul 10, 2016 at 12:32 PM, Zellyn Hunter  wrote:

> Usually not too tricky. The problem is that protos and most other
> programming languages prohibit circular imports at the file level, and Go
> does it at the package level. The fact that protoc and the other languages
> are okay with a topology ensures there is a valid Go repackaging that will
> work.
>
> Zellyn
>
> On Fri, Jul 8, 2016, 6:52 PM 'Josh Haberman' via Protocol Buffers <
> protobuf@googlegroups.com> wrote:
>
>> I don't know the background of the Go import system or go_package option.
>> However this statement concerns me a little:
>>
>> > We have to use go_package to reorganize things that are fine in
>> Java/Ruby, but would be circular package imports in Go.
>>
>> Is this implying that certain .proto files would generate invalid Go code
>> until you manually insert some go_package statements? How tricky is it to
>> do this manual untangling?
>>
>>
>> On Friday, July 1, 2016 at 7:53:03 PM UTC-7, Zellyn wrote:
>>>
>>> Oh wow. Thanks for thinking about it so carefully!
>>>
>>> I should mention: before you could actually specify full package paths
>>> to the Go proto plugin, earlier versions of our wrapper (
>>> https://github.com/square/goprotowrap) just forcibly manhandled things
>>> into the shape we wanted by explicitly specifying -M parameter import
>>> rewrites for *every single* import, and renaming the output files into
>>> place.
>>>
>>> So it's definitely possible to shape things as you want. However, I
>>> would prefer to find a solution that works with the official plugin,
>>> because I don't believe our setup is remarkable or unusual: as GRPC sees an
>>> increase in use as a cross-language RPC mechanism, everyone else is going
>>> to struggle with the same things.
>>>
>>> Zellyn
>>>
>>> On Fri, Jul 1, 2016 at 7:32 PM Ross Light  wrote:
>>>
 Sorry for the delayed response!  I've written about 3-4 draft replies
 over this week that I've discarded because they don't meet your
 requirements.  I'll think about it more over the long weekend and get back
 to you.

 Cheers,
 -Ross


 On Fri, Jul 1, 2016 at 3:11 PM Jie Luo  wrote:

> +Ross who is working on Go protobuf.
>
> Ross, do you have any comments or suggestions?
>
> On Fri, Jun 24, 2016 at 8:35 AM, Zellyn  wrote:
>
>> Hi folks,
>>
>> Apologies in advance for the complex email. It takes a bit of
>> explaining to set up what we're having trouble with.
>>
>> I would like to lay out how we generate protos in Go (using our
>> unfortunately forked version of the protoc plugin), and ask for 
>> suggestions
>> as to how to accomplish the same thing with the unforked version. In the
>> process, I wish to ask questions about the post-vendor-experiment utility
>> of the current behavior of import_prefix, hopefully generating more
>> discussion than I managed to here
>> .
>>
>> Background:
>>
>>- We have protos spread around all over the place: in our Go
>>monorepo, in our Java monorepo, and in miscellaneous Ruby repos.
>>- We collect them all into a single directory before generating
>>Go code. Call it $COLLECT_DIR
>>- We have to use go_package to 

Re: [protobuf] Question about Go protobufs and import_prefix

['Damien Neil' via Protocol Buffers]

Simpler than maintaining a fork, though.

The more convenient alternative would be for protoc to understand Go vendor
directories, but that would require putting some Go-specific logic in
protoc itself.

On Tue, Jul 12, 2016 at 8:00 AM, Zellyn Hunter  wrote:

> Yeah I can do anything with a wrapper. But to the extent that our concerns
> and structure are normal, it would be a shame to need a wrapper.
>
> On Tue, Jul 12, 2016, 8:54 AM Ross Light  wrote:
>
>> Zellyn and I talked a little bit at Gophercon about this. I think some
>> kind of wrapper is in order for his use case: it may actually be a filter
>> between protoc and protoc-gen-go.
>>
>> On Tue, Jul 12, 2016, 8:49 AM Damien Neil  wrote:
>>
>>> If I'm following this correctly, your core problem is that protoc
>>> doesn't understand vendored paths as used by the go tool. For example:
>>>
>>> - You have a proto file located in src/square/up/protos/square.proto.
>>> - square.proto imports "
>>> github.com/golang/protobuf/ptypes/timestamp/timestamp.proto".
>>> - timestamp.proto is actually located in "src/square/up/vendor/
>>> github.com/.../timestamp.proto".
>>>
>>> You can't run "protoc --go_out=. square/up/protos/square.proto", because
>>> protoc won't be able to locate the vendored copy of timestamp.proto. If it
>>> did, everything would work correctly without setting an import_prefix.
>>>
>>> It would be nice for Go users if protoc did understand vendored paths.
>>> Failing that, however, I think you can get what you want with a very small
>>> wrapper around protoc. The wrapper can copy a proto and all its
>>> dependencies into a temp directory (pulling dependencies from vendored
>>> paths as necessary), run protoc there, and copy the result back. e.g., in
>>> the case of the above example:
>>>
>>> $ mkdir -p /tmp/x/square/up/protos
>>> $ cp src/square/up/protos/square.proto /tmp/x/square/up/protos
>>> $ mkdir -p /tmp/x/github.com/golang/protobuf/ptypes/timestamp
>>> $ cp square/up/vendor/
>>> github.com/golang/protobuf/ptypes/timestamp/timestamp.proto
>>> github.com/golang/protobuf/ptypes/timestamp
>>> $ cd /tmp/x && protoc -go_out=. square/up/protos/square.proto
>>> $ cp /tmp/x/square/up/protos/square.pb.go square/up/protos
>>>
>>> The generated .go files will reference non-vendored paths, which the Go
>>> compiler will resolve to the correct vendored directory.
>>>
>>> Does that seem like it would work for you?
>>>
>>> On Sun, Jul 10, 2016 at 12:32 PM, Zellyn Hunter 
>>> wrote:
>>>
 Usually not too tricky. The problem is that protos and most other
 programming languages prohibit circular imports at the file level, and Go
 does it at the package level. The fact that protoc and the other languages
 are okay with a topology ensures there is a valid Go repackaging that will
 work.

 Zellyn

 On Fri, Jul 8, 2016, 6:52 PM 'Josh Haberman' via Protocol Buffers <
 protobuf@googlegroups.com> wrote:

> I don't know the background of the Go import system or go_package
> option. However this statement concerns me a little:
>
> > We have to use go_package to reorganize things that are fine in
> Java/Ruby, but would be circular package imports in Go.
>
> Is this implying that certain .proto files would generate invalid Go
> code until you manually insert some go_package statements? How tricky is 
> it
> to do this manual untangling?
>
>
> On Friday, July 1, 2016 at 7:53:03 PM UTC-7, Zellyn wrote:
>>
>> Oh wow. Thanks for thinking about it so carefully!
>>
>> I should mention: before you could actually specify full package
>> paths to the Go proto plugin, earlier versions of our wrapper (
>> https://github.com/square/goprotowrap) just forcibly manhandled
>> things into the shape we wanted by explicitly specifying -M parameter
>> import rewrites for *every single* import, and renaming the output
>> files into place.
>>
>> So it's definitely possible to shape things as you want. However, I
>> would prefer to find a solution that works with the official plugin,
>> because I don't believe our setup is remarkable or unusual: as GRPC sees 
>> an
>> increase in use as a cross-language RPC mechanism, everyone else is going
>> to struggle with the same things.
>>
>> Zellyn
>>
>> On Fri, Jul 1, 2016 at 7:32 PM Ross Light  wrote:
>>
>>> Sorry for the delayed response!  I've written about 3-4 draft
>>> replies over this week that I've discarded because they don't meet your
>>> requirements.  I'll think about it more over the long weekend and get 
>>> back
>>> to you.
>>>
>>> Cheers,
>>> -Ross
>>>
>>>
>>> On Fri, Jul 1, 2016 at 3:11 PM Jie Luo  wrote:
>>>
 +Ross who is working on Go protobuf.

 Ross, do you have any 

Re: [protobuf] Question about Go protobufs and import_prefix

['Damien Neil' via Protocol Buffers]

(Apologies if you get two copies of this mail; resending to the list 
because my first try bounced.)

If I'm following this correctly, your core problem is that protoc doesn't 
understand vendored paths as used by the go tool. For example:

- You have a proto file located in src/square/up/protos/square.proto.
- square.proto imports 
"github.com/golang/protobuf/ptypes/timestamp/timestamp.proto".
- timestamp.proto is actually located in 
"src/square/up/vendor/github.com/.../timestamp.proto".

You can't run "protoc --go_out=. square/up/protos/square.proto", because 
protoc won't be able to locate the vendored copy of timestamp.proto. If it 
did, everything would work correctly without setting an import_prefix.

It would be nice for Go users if protoc did understand vendored paths. 
Failing that, however, I think you can get what you want with a very small 
wrapper around protoc. The wrapper can copy a proto and all its 
dependencies into a temp directory (pulling dependencies from vendored 
paths as necessary), run protoc there, and copy the result back. e.g., in 
the case of the above example:

$ mkdir -p /tmp/x/square/up/protos
$ cp src/square/up/protos/square.proto /tmp/x/square/up/protos
$ mkdir -p /tmp/x/github.com/golang/protobuf/ptypes/timestamp
$ cp 
square/up/vendor/github.com/golang/protobuf/ptypes/timestamp/timestamp.proto 
github.com/golang/protobuf/ptypes/timestamp
$ cd /tmp/x && protoc -go_out=. square/up/protos/square.proto
$ cp /tmp/x/square/up/protos/square.pb.go square/up/protos

The generated .go files will reference non-vendored paths, which the Go 
compiler will resolve to the correct vendored directory.

Does that seem like it would work for you?


On Sunday, July 10, 2016 at 12:32:26 PM UTC-7, Zellyn wrote:
>
> Usually not too tricky. The problem is that protos and most other 
> programming languages prohibit circular imports at the file level, and Go 
> does it at the package level. The fact that protoc and the other languages 
> are okay with a topology ensures there is a valid Go repackaging that will 
> work.
>
> Zellyn
>
> On Fri, Jul 8, 2016, 6:52 PM 'Josh Haberman' via Protocol Buffers <
> prot...@googlegroups.com > wrote:
>
>> I don't know the background of the Go import system or go_package option. 
>> However this statement concerns me a little:
>>
>> > We have to use go_package to reorganize things that are fine in 
>> Java/Ruby, but would be circular package imports in Go.
>>
>> Is this implying that certain .proto files would generate invalid Go code 
>> until you manually insert some go_package statements? How tricky is it to 
>> do this manual untangling?
>>
>>
>> On Friday, July 1, 2016 at 7:53:03 PM UTC-7, Zellyn wrote:
>>>
>>> Oh wow. Thanks for thinking about it so carefully!
>>>
>>> I should mention: before you could actually specify full package paths 
>>> to the Go proto plugin, earlier versions of our wrapper (
>>> https://github.com/square/goprotowrap) just forcibly manhandled things 
>>> into the shape we wanted by explicitly specifying -M parameter import 
>>> rewrites for *every single* import, and renaming the output files into 
>>> place.
>>>
>>> So it's definitely possible to shape things as you want. However, I 
>>> would prefer to find a solution that works with the official plugin, 
>>> because I don't believe our setup is remarkable or unusual: as GRPC sees an 
>>> increase in use as a cross-language RPC mechanism, everyone else is going 
>>> to struggle with the same things.
>>>
>>> Zellyn
>>>
>>> On Fri, Jul 1, 2016 at 7:32 PM Ross Light >> > wrote:
>>>
 Sorry for the delayed response!  I've written about 3-4 draft replies 
 over this week that I've discarded because they don't meet your 
 requirements.  I'll think about it more over the long weekend and get back 
 to you.

 Cheers,
 -Ross


 On Fri, Jul 1, 2016 at 3:11 PM Jie Luo  
 wrote:

> +Ross who is working on Go protobuf.
>
> Ross, do you have any comments or suggestions?
>
> On Fri, Jun 24, 2016 at 8:35 AM, Zellyn  > wrote:
>
>> Hi folks,
>>
>> Apologies in advance for the complex email. It takes a bit of 
>> explaining to set up what we're having trouble with.
>>
>> I would like to lay out how we generate protos in Go (using our 
>> unfortunately forked version of the protoc plugin), and ask for 
>> suggestions 
>> as to how to accomplish the same thing with the unforked version. In the 
>> process, I wish to ask questions about the post-vendor-experiment 
>> utility 
>> of the current behavior of import_prefix, hopefully generating more 
>> discussion than I managed to here 
>> .
>>
>> Background:
>>
>>- We have protos spread around all over the place: in our Go 
>>monorepo, in our Java monorepo, and in miscellaneous Ruby 

Re: [protobuf] Question about Go protobufs and import_prefix

[Zellyn Hunter]

Yeah I can do anything with a wrapper. But to the extent that our concerns
and structure are normal, it would be a shame to need a wrapper.

On Tue, Jul 12, 2016, 8:54 AM Ross Light  wrote:

> Zellyn and I talked a little bit at Gophercon about this. I think some
> kind of wrapper is in order for his use case: it may actually be a filter
> between protoc and protoc-gen-go.
>
> On Tue, Jul 12, 2016, 8:49 AM Damien Neil  wrote:
>
>> If I'm following this correctly, your core problem is that protoc doesn't
>> understand vendored paths as used by the go tool. For example:
>>
>> - You have a proto file located in src/square/up/protos/square.proto.
>> - square.proto imports "
>> github.com/golang/protobuf/ptypes/timestamp/timestamp.proto".
>> - timestamp.proto is actually located in "src/square/up/vendor/
>> github.com/.../timestamp.proto".
>>
>> You can't run "protoc --go_out=. square/up/protos/square.proto", because
>> protoc won't be able to locate the vendored copy of timestamp.proto. If it
>> did, everything would work correctly without setting an import_prefix.
>>
>> It would be nice for Go users if protoc did understand vendored paths.
>> Failing that, however, I think you can get what you want with a very small
>> wrapper around protoc. The wrapper can copy a proto and all its
>> dependencies into a temp directory (pulling dependencies from vendored
>> paths as necessary), run protoc there, and copy the result back. e.g., in
>> the case of the above example:
>>
>> $ mkdir -p /tmp/x/square/up/protos
>> $ cp src/square/up/protos/square.proto /tmp/x/square/up/protos
>> $ mkdir -p /tmp/x/github.com/golang/protobuf/ptypes/timestamp
>> $ cp square/up/vendor/
>> github.com/golang/protobuf/ptypes/timestamp/timestamp.proto
>> github.com/golang/protobuf/ptypes/timestamp
>> $ cd /tmp/x && protoc -go_out=. square/up/protos/square.proto
>> $ cp /tmp/x/square/up/protos/square.pb.go square/up/protos
>>
>> The generated .go files will reference non-vendored paths, which the Go
>> compiler will resolve to the correct vendored directory.
>>
>> Does that seem like it would work for you?
>>
>> On Sun, Jul 10, 2016 at 12:32 PM, Zellyn Hunter  wrote:
>>
>>> Usually not too tricky. The problem is that protos and most other
>>> programming languages prohibit circular imports at the file level, and Go
>>> does it at the package level. The fact that protoc and the other languages
>>> are okay with a topology ensures there is a valid Go repackaging that will
>>> work.
>>>
>>> Zellyn
>>>
>>> On Fri, Jul 8, 2016, 6:52 PM 'Josh Haberman' via Protocol Buffers <
>>> protobuf@googlegroups.com> wrote:
>>>
 I don't know the background of the Go import system or go_package
 option. However this statement concerns me a little:

 > We have to use go_package to reorganize things that are fine in
 Java/Ruby, but would be circular package imports in Go.

 Is this implying that certain .proto files would generate invalid Go
 code until you manually insert some go_package statements? How tricky is it
 to do this manual untangling?


 On Friday, July 1, 2016 at 7:53:03 PM UTC-7, Zellyn wrote:
>
> Oh wow. Thanks for thinking about it so carefully!
>
> I should mention: before you could actually specify full package paths
> to the Go proto plugin, earlier versions of our wrapper (
> https://github.com/square/goprotowrap) just forcibly manhandled
> things into the shape we wanted by explicitly specifying -M parameter
> import rewrites for *every single* import, and renaming the output
> files into place.
>
> So it's definitely possible to shape things as you want. However, I
> would prefer to find a solution that works with the official plugin,
> because I don't believe our setup is remarkable or unusual: as GRPC sees 
> an
> increase in use as a cross-language RPC mechanism, everyone else is going
> to struggle with the same things.
>
> Zellyn
>
> On Fri, Jul 1, 2016 at 7:32 PM Ross Light  wrote:
>
>> Sorry for the delayed response!  I've written about 3-4 draft replies
>> over this week that I've discarded because they don't meet your
>> requirements.  I'll think about it more over the long weekend and get 
>> back
>> to you.
>>
>> Cheers,
>> -Ross
>>
>>
>> On Fri, Jul 1, 2016 at 3:11 PM Jie Luo  wrote:
>>
>>> +Ross who is working on Go protobuf.
>>>
>>> Ross, do you have any comments or suggestions?
>>>
>>> On Fri, Jun 24, 2016 at 8:35 AM, Zellyn  wrote:
>>>
 Hi folks,

 Apologies in advance for the complex email. It takes a bit of
 explaining to set up what we're having trouble with.

 I would like to lay out how we generate protos in Go (using our
 unfortunately forked version of