[protobuf] Need Help on Cross compilation of GRPC.

[Neeli Srinivas]

Hi All,

This is Srinivas.  I am new to GRPC, I am trying to compile latest grpc 
(v1.7.2)  version.
It's working in X86 architecture. when i am trying to compile for ARM 
architecture(cortex A9) it's not working.
Please guide me the steps. One more thing  i am using protobuf 3.4.0 
version. I am going to explain what i followed in below.

followed steps:
-- 
Environment Details:
1)linux 14.04 
2)uname -a :
Linux aaa 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18 11:55:51 UTC 
2017 x86_64 x86_64 x86_64 GNU/Linux

steps:1
-

-> Compiled for x86 
-> GRPC version V1.7.2
-> Protobuf 3.4.0
-> Compilation was success 

$ ./configure
$ make
$ make check
$ sudo make install
$ sudo ldconfig # refresh shared library cache.

 -> followed this link  
https://github.com/google/protobuf/blob/master/src/README.md
 
Steps:2 (Trying to compile for ARM by using cross compiler)
-- 
-

- Using cross compilers 
 arm-linux-gnueabi-gcc
 arm-linux-gnueabi-g++.
-> cd grpc
-> git submodule update --init
-> ./autogen.sh
-> ./configure  --host=arm-linux 
--with-protoc=/home//grpc/third_party/protobuf/COMP/bin/protoc 
--prefix=/home//grpc/third_party/protobuf/build
-> make 
-> make check
-> sudo make install


In this process i am getting in errors. while doing sudo make install 
facing issues. 

** Here My question is ?
 - What i am following is it correct or not ?
 - If my procedure wrong . please share me the correct procedure. 
-Advanced thanks




Thanks
Srinivas Neeli



 

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] How to calculate protobuf size?

[Brian Palmer]

The simplest way would be to serialize your messages into bytes and then
measure the size directly, right?

On Sun, Dec 10, 2017 at 11:26 AM, Tomer Praizler 
wrote:

> Couldn't find a nice and easy way to calculate my protobuf sizes using
> python. Is there a recommended way?
>
> I am sending my protobufs into rabbitmq, and huge messages causing my
> connection to break. I want to be able to validate the size before sending
> a message.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to protobuf+unsubscr...@googlegroups.com.
> To post to this group, send email to protobuf@googlegroups.com.
> Visit this group at https://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[protobuf] Re: Errors during make install.

[maicah]

I have a similar problem: after successfully downloading and un-taring 
protobuf-all-3.5.0.tar.bz, I also successfully ran  ."/confiure", "make" 
and "make check". Then "sudo make install"  failed with:

libtool: install: (cd /downloads/protobuf-3.5.0/src; /bin/sh 
/downloads/protobuf-3.5.0/libtool  --tag CXX --mode=relink g++ 
-pthread -DHAVE_PTHREAD=1 -DHAVE_ZLIB=1 -Wall -Wno-sign-compare -O2 -g 
-DNDEBUG -version-info 15:0:0 -export-dynamic -no-undefined 
-Wl,--version-script=./libprotoc.map -o libprotoc.la -rpath /usr/local/lib 
google/protobuf/compiler/code_generator.lo 
google/protobuf/compiler/command_line_interface.lo 
google/protobuf/compiler/plugin.lo google/protobuf/compiler/plugin.pb.lo 
google/protobuf/compiler/subprocess.lo 
google/protobuf/compiler/zip_writer.lo 
google/protobuf/compiler/cpp/cpp_enum.lo 
google/protobuf/compiler/cpp/cpp_enum_field.lo goo ===snip==
base.lo google/protobuf/compiler/csharp/csharp_wrapper_field.lo -lpthread 
libprotobuf.la -lz )
mv: cannot move 'libprotoc.so.15.0.0' to 'libprotoc.so.15.0.0U': Permission 
denied
libtool: install: error: relink `libprotoc.la' with the above command 
before installing it
make[3]: *** [install-libLTLIBRARIES] Error 1

It is a permission problem.

On Thursday, January 7, 2010 at 5:37:06 PM UTC-5, mancha wrote:
>
> I get these error during make install. make and make check suceeded
> earlier.
> Can you please advise how to resolve this.
>
> Thanks,
> Madhav.
>
> /usr/bin/ld: /usr/local/lib/libz.a(deflate.o): relocation R_X86_64_32S
> against `.rodata' can not be used when making a shared object;
> recompile with -fPIC
> /usr/local/lib/libz.a: could not read symbols: Bad value
> collect2: ld returned 1 exit status
> libtool: install: error: relink `libprotoc.la' with the above command
> before installing it
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[protobuf] How to calculate protobuf size?

[Tomer Praizler]

Couldn't find a nice and easy way to calculate my protobuf sizes using 
python. Is there a recommended way?

I am sending my protobufs into rabbitmq, and huge messages causing my 
connection to break. I want to be able to validate the size before sending 
a message.

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] Re: validating protobuf messages

[Marc Gravell]

a zero tag is never valid in any protobuf data, although it wouldn't be
unheard of for folks to use a zero tag as a sentinel value to demark
multiple root messages. Protoc has some facilities to check the insides of
a message that might help you figure out how likely it is to be a match,
but it won't make a zero tag work.

On 10 Dec 2017 6:35 p.m., "Annu Bansal"  wrote:

> Hey Jeroen!
> Dis you get any method to check if a .pb file is generated from givem
> .proto file? Actually, I'm getting error while deserializing a .pb file,
> "The protocol message contained invalid tag(zero)." I was wondering if the
> two files are compatible.
> Thanks in advance.
>
> On Saturday, November 17, 2012 at 5:38:16 AM UTC+5:30, Jeroen Ooms wrote:
>>
>> I am using the RProtoBuf package, which interfaces to the c++ protobuf
>> library. I was wondering if there is any way of 'validating' a pb
>> message, in the sense that you can check if it actually is a valid
>> message for a given pb description. Currently, RProtoBuf is very
>> permissive and will basically accept anything. E.g the code below will
>> not throw an error, even though the file obviously is not a valid
>> tutorial.Person message:
>>
>> library(RProtoBuf)
>> read(tutorial.Person, "/etc/passwd")
>>
>> In practice, it is quite easy to introduce errors by accidentally reading
>> a message using the wrong description. It would be very helpful if there
>> was a way of preventing this kind of problems by some sort of validation.
>>
>> Is there support for this in the c++ library, or does protobuf basically
>> assume that the user/application has some other way of validating a message?
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to protobuf+unsubscr...@googlegroups.com.
> To post to this group, send email to protobuf@googlegroups.com.
> Visit this group at https://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[protobuf] Re: validating protobuf messages

[Annu Bansal]

Hey Jeroen!
Dis you get any method to check if a .pb file is generated from givem 
.proto file? Actually, I'm getting error while deserializing a .pb file, 
"The protocol message contained invalid tag(zero)." I was wondering if the 
two files are compatible. 
Thanks in advance. 

On Saturday, November 17, 2012 at 5:38:16 AM UTC+5:30, Jeroen Ooms wrote:
>
> I am using the RProtoBuf package, which interfaces to the c++ protobuf 
> library. I was wondering if there is any way of 'validating' a pb 
> message, in the sense that you can check if it actually is a valid 
> message for a given pb description. Currently, RProtoBuf is very 
> permissive and will basically accept anything. E.g the code below will 
> not throw an error, even though the file obviously is not a valid 
> tutorial.Person message:
>
> library(RProtoBuf)
> read(tutorial.Person, "/etc/passwd")
>
> In practice, it is quite easy to introduce errors by accidentally reading 
> a message using the wrong description. It would be very helpful if there 
> was a way of preventing this kind of problems by some sort of validation.
>
> Is there support for this in the c++ library, or does protobuf basically 
> assume that the user/application has some other way of validating a message?
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] Re: Any way to "extract all" using protoc or any other tool?

[Ilia Mirkin]

On Sun, Dec 10, 2017 at 11:23 AM, Jim Baldwin  wrote:
> Perhaps it might help if I understood the output of protoc --decode_raw.
>
> Here's an example of a .caffemodel file I'm trying to inspect.  Is there a
> description of what the numbers mean in this file?

As I said... it's a sequence of (tag, value) pairs, so...

>
> 1: "VGG_ILSVRC_16_layers"

Tag id 1: string value "VGG_bla"

> 100 {

Tag id 100: sub-message of some length (not printed here, but it's in
the encoded proto)

>   1: "input-data"

Tag id 1 of sub-message: "input-data"

(I won't annotate the rest, hopefully you get the gist of it.)

So for example, you might have a proto which is

message foo {
  string a = 1;
  repeated message bar = 100;
};

that would lead to that sort of encoding. Nowhere is type information
encoded on the wire unless you explciitly put it there yourself.
There's no way to know which proto it is, and there can be any number
of protos that would yield a valid decoding. So you have to give it
the name of the proto it is with --decode.

Protobuf is a structure encoding/decoding mechanism. Anything extra,
like type names or framing have to come from surrounding logic.

  -ilia

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] Any way to "extract all" using protoc or any other tool?

[Jim Baldwin]

Hmmm. Back to the core problem.  I am not certain I have the root.

Case in point:  
caffe.proto:  
https://github.com/BVLC/caffe/blob/master/src/caffe/proto/caffe.proto

I'm gradually thinking the root of this file is "NetParameter", but I'm not 
sure how I was supposed to know that. :-P

I still think there should be a tool that can do (protobuf, .proto) <-> 
JSON.

On Sunday, December 10, 2017 at 8:45:13 AM UTC-8, Marc Gravell wrote:
>
> oh, there is a root. if i had to guess it has something like:
>
>
> message TheRootMessageType {
> required string name = 1;
> repeated SomeNoun items = 100;
> }
>
>
>
> On 10 Dec 2017 4:42 p.m., "Jim Baldwin"  
> wrote:
>
>> In this case, it looks like there isn’t a root.  Rather, this format is a 
>> series of top-level parameters.  So, I have to give it the Parameter I’m 
>> looking for.  The problem I have with this is the order of parameters 
>> _might_ matter, and I lose that by only looking for one.
>>
>> On Dec 10, 2017, at 8:35 AM, Marc Gravell > > wrote:
>>
>> as I ubderstand it, --decode *will do that*. it doesn't decode *just* the 
>> root : but, it needs to know the root message type in order to correctly 
>> interpret the data
>>
>> On 10 Dec 2017 4:32 p.m., "Jim Baldwin"  
>> wrote:
>>
>>> OK, this helps.  I need to figure out what the "root" message is.  It 
>>> seems like an omission in the whole PB thing that you can't specify the 
>>> .proto and do a --decode_everything.
>>>
>>> On Sunday, December 10, 2017 at 8:23:12 AM UTC-8, Marc Gravell wrote: 

 You can and it does. The problem is that the wire format by itself 
 doesn't tell it **what message type** the root object is. So you need to 
 tell it in the additional parameter to --decode

 On 10 Dec 2017 3:14 p.m., "Jim Baldwin"  wrote:

 It's not really just a sequence; it's a hierarchy, isn't it?  Why can't 
 I use --decode  or something like that?


 On Saturday, December 9, 2017 at 4:20:15 PM UTC-8, Ilia Mirkin wrote: 

> On Sat, Dec 9, 2017 at 5:52 PM, Jim Baldwin  wrote: 
> > I have a protobuf file, and a .proto file that describes the schema.
>  
> > 
> > The .proto describes dozens of different messages that may be in the
>  
> > protobuf file. 
> > 
> > I would like to know what messages can be found in the file.  I do a 
> protoc 
> > --decode_raw and get something out, but I don't see how to use that 
> to 
> > figure out how to extract messages from the file. 
> > 
> > I assume there's something I don't get about protobufs, but it seems 
> to me I 
> > should be able to take a protobuf data file and corresponding .proto 
> and 
> > turn it into a file that lets me see what the message hierarchy is 
> in the 
> > file.  JSON would be a great way to do that. 
> > 
> > What am I missing? 
>
> An encoded protobuf is just a sequence of (tag, value) pairs. If you 
> don't know which proto it is, decode_raw is the best you can do. If 
> you do know which proto it is, you can use --decode instead and pass 
> it a proto name to use for the decoding. 
>
> Cheers, 
>
>   -ilia 
>

 -- 
 You received this message because you are subscribed to the Google 
 Groups "Protocol Buffers" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to protobuf+u...@googlegroups.com.
 To post to this group, send email to prot...@googlegroups.com.
 Visit this group at https://groups.google.com/group/protobuf.
 For more options, visit https://groups.google.com/d/optout.



>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Protocol Buffers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to protobuf+u...@googlegroups.com .
>>> To post to this group, send email to prot...@googlegroups.com 
>>> .
>>> Visit this group at https://groups.google.com/group/protobuf.
>>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] Any way to "extract all" using protoc or any other tool?

[Marc Gravell]

oh, there is a root. if i had to guess it has something like:


message TheRootMessageType {
required string name = 1;
repeated SomeNoun items = 100;
}



On 10 Dec 2017 4:42 p.m., "Jim Baldwin"  wrote:

> In this case, it looks like there isn’t a root.  Rather, this format is a
> series of top-level parameters.  So, I have to give it the Parameter I’m
> looking for.  The problem I have with this is the order of parameters
> _might_ matter, and I lose that by only looking for one.
>
> On Dec 10, 2017, at 8:35 AM, Marc Gravell  wrote:
>
> as I ubderstand it, --decode *will do that*. it doesn't decode *just* the
> root : but, it needs to know the root message type in order to correctly
> interpret the data
>
> On 10 Dec 2017 4:32 p.m., "Jim Baldwin"  wrote:
>
>> OK, this helps.  I need to figure out what the "root" message is.  It
>> seems like an omission in the whole PB thing that you can't specify the
>> .proto and do a --decode_everything.
>>
>> On Sunday, December 10, 2017 at 8:23:12 AM UTC-8, Marc Gravell wrote:
>>>
>>> You can and it does. The problem is that the wire format by itself
>>> doesn't tell it **what message type** the root object is. So you need to
>>> tell it in the additional parameter to --decode
>>>
>>> On 10 Dec 2017 3:14 p.m., "Jim Baldwin"  wrote:
>>>
>>> It's not really just a sequence; it's a hierarchy, isn't it?  Why can't
>>> I use --decode  or something like that?
>>>
>>>
>>> On Saturday, December 9, 2017 at 4:20:15 PM UTC-8, Ilia Mirkin wrote:
>>>
 On Sat, Dec 9, 2017 at 5:52 PM, Jim Baldwin  wrote:
 > I have a protobuf file, and a .proto file that describes the schema.
 >
 > The .proto describes dozens of different messages that may be in the
 > protobuf file.
 >
 > I would like to know what messages can be found in the file.  I do a
 protoc
 > --decode_raw and get something out, but I don't see how to use that to

 > figure out how to extract messages from the file.
 >
 > I assume there's something I don't get about protobufs, but it seems
 to me I
 > should be able to take a protobuf data file and corresponding .proto
 and
 > turn it into a file that lets me see what the message hierarchy is in
 the
 > file.  JSON would be a great way to do that.
 >
 > What am I missing?

 An encoded protobuf is just a sequence of (tag, value) pairs. If you
 don't know which proto it is, decode_raw is the best you can do. If
 you do know which proto it is, you can use --decode instead and pass
 it a proto name to use for the decoding.

 Cheers,

   -ilia

>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Protocol Buffers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to protobuf+u...@googlegroups.com.
>>> To post to this group, send email to prot...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/protobuf.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Protocol Buffers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to protobuf+unsubscr...@googlegroups.com.
>> To post to this group, send email to protobuf@googlegroups.com.
>> Visit this group at https://groups.google.com/group/protobuf.
>> For more options, visit https://groups.google.com/d/optout.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] Any way to "extract all" using protoc or any other tool?

[Jim Baldwin]

In this case, it looks like there isn’t a root.  Rather, this format is a 
series of top-level parameters.  So, I have to give it the Parameter I’m 
looking for.  The problem I have with this is the order of parameters _might_ 
matter, and I lose that by only looking for one.

On Dec 10, 2017, at 8:35 AM, Marc Gravell 
> wrote:

as I ubderstand it, --decode *will do that*. it doesn't decode *just* the root 
: but, it needs to know the root message type in order to correctly interpret 
the data

On 10 Dec 2017 4:32 p.m., "Jim Baldwin" 
> wrote:
OK, this helps.  I need to figure out what the "root" message is.  It seems 
like an omission in the whole PB thing that you can't specify the .proto and do 
a --decode_everything.

On Sunday, December 10, 2017 at 8:23:12 AM UTC-8, Marc Gravell wrote:
You can and it does. The problem is that the wire format by itself doesn't tell 
it **what message type** the root object is. So you need to tell it in the 
additional parameter to --decode

On 10 Dec 2017 3:14 p.m., "Jim Baldwin"  wrote:
It's not really just a sequence; it's a hierarchy, isn't it?  Why can't I use 
--decode  or something like that?


On Saturday, December 9, 2017 at 4:20:15 PM UTC-8, Ilia Mirkin wrote:
On Sat, Dec 9, 2017 at 5:52 PM, Jim Baldwin  wrote:
> I have a protobuf file, and a .proto file that describes the schema.
>
> The .proto describes dozens of different messages that may be in the
> protobuf file.
>
> I would like to know what messages can be found in the file.  I do a protoc
> --decode_raw and get something out, but I don't see how to use that to
> figure out how to extract messages from the file.
>
> I assume there's something I don't get about protobufs, but it seems to me I
> should be able to take a protobuf data file and corresponding .proto and
> turn it into a file that lets me see what the message hierarchy is in the
> file.  JSON would be a great way to do that.
>
> What am I missing?

An encoded protobuf is just a sequence of (tag, value) pairs. If you
don't know which proto it is, decode_raw is the best you can do. If
you do know which proto it is, you can use --decode instead and pass
it a proto name to use for the decoding.

Cheers,

  -ilia

--
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+u...@googlegroups.com.
To post to this group, send email to prot...@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 
protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to 
protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] Any way to "extract all" using protoc or any other tool?

[Marc Gravell]

as I ubderstand it, --decode *will do that*. it doesn't decode *just* the
root : but, it needs to know the root message type in order to correctly
interpret the data

On 10 Dec 2017 4:32 p.m., "Jim Baldwin"  wrote:

> OK, this helps.  I need to figure out what the "root" message is.  It
> seems like an omission in the whole PB thing that you can't specify the
> .proto and do a --decode_everything.
>
> On Sunday, December 10, 2017 at 8:23:12 AM UTC-8, Marc Gravell wrote:
>>
>> You can and it does. The problem is that the wire format by itself
>> doesn't tell it **what message type** the root object is. So you need to
>> tell it in the additional parameter to --decode
>>
>> On 10 Dec 2017 3:14 p.m., "Jim Baldwin"  wrote:
>>
>> It's not really just a sequence; it's a hierarchy, isn't it?  Why can't I
>> use --decode  or something like that?
>>
>>
>> On Saturday, December 9, 2017 at 4:20:15 PM UTC-8, Ilia Mirkin wrote:
>>
>>> On Sat, Dec 9, 2017 at 5:52 PM, Jim Baldwin  wrote:
>>> > I have a protobuf file, and a .proto file that describes the schema.
>>> >
>>> > The .proto describes dozens of different messages that may be in the
>>> > protobuf file.
>>> >
>>> > I would like to know what messages can be found in the file.  I do a
>>> protoc
>>> > --decode_raw and get something out, but I don't see how to use that to
>>> > figure out how to extract messages from the file.
>>> >
>>> > I assume there's something I don't get about protobufs, but it seems
>>> to me I
>>> > should be able to take a protobuf data file and corresponding .proto
>>> and
>>> > turn it into a file that lets me see what the message hierarchy is in
>>> the
>>> > file.  JSON would be a great way to do that.
>>> >
>>> > What am I missing?
>>>
>>> An encoded protobuf is just a sequence of (tag, value) pairs. If you
>>> don't know which proto it is, decode_raw is the best you can do. If
>>> you do know which proto it is, you can use --decode instead and pass
>>> it a proto name to use for the decoding.
>>>
>>> Cheers,
>>>
>>>   -ilia
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Protocol Buffers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to protobuf+u...@googlegroups.com.
>> To post to this group, send email to prot...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/protobuf.
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to protobuf+unsubscr...@googlegroups.com.
> To post to this group, send email to protobuf@googlegroups.com.
> Visit this group at https://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] Any way to "extract all" using protoc or any other tool?

[Jim Baldwin]

OK, this helps.  I need to figure out what the "root" message is.  It seems 
like an omission in the whole PB thing that you can't specify the .proto 
and do a --decode_everything.

On Sunday, December 10, 2017 at 8:23:12 AM UTC-8, Marc Gravell wrote:
>
> You can and it does. The problem is that the wire format by itself doesn't 
> tell it **what message type** the root object is. So you need to tell it in 
> the additional parameter to --decode
>
> On 10 Dec 2017 3:14 p.m., "Jim Baldwin"  
> wrote:
>
> It's not really just a sequence; it's a hierarchy, isn't it?  Why can't I 
> use --decode  or something like that?
>
>
> On Saturday, December 9, 2017 at 4:20:15 PM UTC-8, Ilia Mirkin wrote:
>
>> On Sat, Dec 9, 2017 at 5:52 PM, Jim Baldwin  wrote: 
>> > I have a protobuf file, and a .proto file that describes the schema. 
>> > 
>> > The .proto describes dozens of different messages that may be in the 
>> > protobuf file. 
>> > 
>> > I would like to know what messages can be found in the file.  I do a 
>> protoc 
>> > --decode_raw and get something out, but I don't see how to use that to 
>> > figure out how to extract messages from the file. 
>> > 
>> > I assume there's something I don't get about protobufs, but it seems to 
>> me I 
>> > should be able to take a protobuf data file and corresponding .proto 
>> and 
>> > turn it into a file that lets me see what the message hierarchy is in 
>> the 
>> > file.  JSON would be a great way to do that. 
>> > 
>> > What am I missing? 
>>
>> An encoded protobuf is just a sequence of (tag, value) pairs. If you 
>> don't know which proto it is, decode_raw is the best you can do. If 
>> you do know which proto it is, you can use --decode instead and pass 
>> it a proto name to use for the decoding. 
>>
>> Cheers, 
>>
>>   -ilia 
>>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to protobuf+u...@googlegroups.com .
> To post to this group, send email to prot...@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] Re: Any way to "extract all" using protoc or any other tool?

[Marc Gravell]

They are field numbers. They don't mean anything by themselves other than
to identify each field. If you want to know the logical *name* of each
field, you need the .proto schema.

On 10 Dec 2017 4:23 p.m., "Jim Baldwin"  wrote:

> Perhaps it might help if I understood the output of protoc --decode_raw.
>
> Here's an example of a .caffemodel file I'm trying to inspect.  Is there a
> description of what the numbers mean in this file?
>
> 1: "VGG_ILSVRC_16_layers"
> 100 {
>   1: "input-data"
>   2: "Python"
>   4: "data"
>   4: "im_info"
>   4: "gt_boxes"
>   10: 0
>   130 {
> 1: "roi_data_layer.layer"
> 2: "RoIDataLayer"
> 3: "\'num_classes\': 2"
>   }
> }
> 100 {
>   1: "data_input-data_0_split"
>   2: "Split"
>   3: "data"
>   4: "data_input-data_0_split_0"
>   4: "data_input-data_0_split_1"
>   10: 0
> }
>
> On Saturday, December 9, 2017 at 2:52:10 PM UTC-8, Jim Baldwin wrote:
>>
>> I have a protobuf file, and a .proto file that describes the schema.
>>
>> The .proto describes dozens of different messages that may be in the
>> protobuf file.
>>
>> I would like to know what messages can be found in the file.  I do a
>> protoc --decode_raw and get something out, but I don't see how to use that
>> to figure out how to extract messages from the file.
>>
>> I assume there's something I don't get about protobufs, but it seems to
>> me I should be able to take a protobuf data file and corresponding .proto
>> and turn it into a file that lets me see what the message hierarchy is in
>> the file.  JSON would be a great way to do that.
>>
>> What am I missing?
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to protobuf+unsubscr...@googlegroups.com.
> To post to this group, send email to protobuf@googlegroups.com.
> Visit this group at https://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[protobuf] Re: Any way to "extract all" using protoc or any other tool?

[Jim Baldwin]

Perhaps it might help if I understood the output of protoc --decode_raw. 

Here's an example of a .caffemodel file I'm trying to inspect.  Is there a 
description of what the numbers mean in this file?

1: "VGG_ILSVRC_16_layers"
100 {
  1: "input-data"
  2: "Python"
  4: "data"
  4: "im_info"
  4: "gt_boxes"
  10: 0
  130 {
1: "roi_data_layer.layer"
2: "RoIDataLayer"
3: "\'num_classes\': 2"
  }
}
100 {
  1: "data_input-data_0_split"
  2: "Split"
  3: "data"
  4: "data_input-data_0_split_0"
  4: "data_input-data_0_split_1"
  10: 0
}

On Saturday, December 9, 2017 at 2:52:10 PM UTC-8, Jim Baldwin wrote:
>
> I have a protobuf file, and a .proto file that describes the schema.
>
> The .proto describes dozens of different messages that may be in the 
> protobuf file.
>
> I would like to know what messages can be found in the file.  I do a 
> protoc --decode_raw and get something out, but I don't see how to use that 
> to figure out how to extract messages from the file.
>
> I assume there's something I don't get about protobufs, but it seems to me 
> I should be able to take a protobuf data file and corresponding .proto and 
> turn it into a file that lets me see what the message hierarchy is in the 
> file.  JSON would be a great way to do that.
>
> What am I missing?
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] Any way to "extract all" using protoc or any other tool?

[Marc Gravell]

You can and it does. The problem is that the wire format by itself doesn't
tell it **what message type** the root object is. So you need to tell it in
the additional parameter to --decode

On 10 Dec 2017 3:14 p.m., "Jim Baldwin"  wrote:

It's not really just a sequence; it's a hierarchy, isn't it?  Why can't I
use --decode  or something like that?


On Saturday, December 9, 2017 at 4:20:15 PM UTC-8, Ilia Mirkin wrote:

> On Sat, Dec 9, 2017 at 5:52 PM, Jim Baldwin  wrote:
> > I have a protobuf file, and a .proto file that describes the schema.
> >
> > The .proto describes dozens of different messages that may be in the
> > protobuf file.
> >
> > I would like to know what messages can be found in the file.  I do a
> protoc
> > --decode_raw and get something out, but I don't see how to use that to
> > figure out how to extract messages from the file.
> >
> > I assume there's something I don't get about protobufs, but it seems to
> me I
> > should be able to take a protobuf data file and corresponding .proto and
> > turn it into a file that lets me see what the message hierarchy is in
> the
> > file.  JSON would be a great way to do that.
> >
> > What am I missing?
>
> An encoded protobuf is just a sequence of (tag, value) pairs. If you
> don't know which proto it is, decode_raw is the best you can do. If
> you do know which proto it is, you can use --decode instead and pass
> it a proto name to use for the decoding.
>
> Cheers,
>
>   -ilia
>
-- 
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] Any way to "extract all" using protoc or any other tool?

[Jim Baldwin]

It's not really just a sequence; it's a hierarchy, isn't it?  Why can't I 
use --decode  or something like that?


On Saturday, December 9, 2017 at 4:20:15 PM UTC-8, Ilia Mirkin wrote:
>
> On Sat, Dec 9, 2017 at 5:52 PM, Jim Baldwin  > wrote: 
> > I have a protobuf file, and a .proto file that describes the schema. 
> > 
> > The .proto describes dozens of different messages that may be in the 
> > protobuf file. 
> > 
> > I would like to know what messages can be found in the file.  I do a 
> protoc 
> > --decode_raw and get something out, but I don't see how to use that to 
> > figure out how to extract messages from the file. 
> > 
> > I assume there's something I don't get about protobufs, but it seems to 
> me I 
> > should be able to take a protobuf data file and corresponding .proto and 
> > turn it into a file that lets me see what the message hierarchy is in 
> the 
> > file.  JSON would be a great way to do that. 
> > 
> > What am I missing? 
>
> An encoded protobuf is just a sequence of (tag, value) pairs. If you 
> don't know which proto it is, decode_raw is the best you can do. If 
> you do know which proto it is, you can use --decode instead and pass 
> it a proto name to use for the decoding. 
>
> Cheers, 
>
>   -ilia 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.