[protobuf] Golang Dynamically create Message from .proto files

['Gary Vidal' via Protocol Buffers]

I have a use case where I need to dynamically read a .proto definition and 
deserialize a byte array of matching message. I scoured the internet does 
anyone have a working example. I am writing a kafka plugin that filters 
messages and uses a schema registry to store and retrieve proto files.  

Something like the following:
DeserializeMessage(proto string, bytes byte[]). 

Must be in go please.

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/protobuf/5319742e-b3f3-4d02-aaa3-e99836138914n%40googlegroups.com.

[protobuf] Unclearness in the specification

[Mattias Hansson]

I am currently trying to implement a protobuf parser but it's a bit unclear 
to me how I should interpret some parts of the specification.

Lets start with RPC option. Options for normal fields and enum fields are 
pretty clear.

Here's the specification for normal fields:
field = [ "repeated" ] type fieldName "=" fieldNumber [ "[" fieldOptions 
"]" ] ";" fieldOptions = fieldOption { "," fieldOption } fieldOption = 
optionName "=" constant

Here's the specification for enum fields:
enum = "enum" enumName enumBody enumBody = "{" { option | enumField | 
emptyStatement | reserved } "}" enumField = ident "=" [ "-" ] intLit [ "[" 
enumValueOption { "," enumValueOption } "]" ]";" enumValueOption = 
optionName "=" constant

Both fields boils down to optionName which is specified as:
optionName = ( ident | "(" ["."] fullIdent ")" )  
Here's the specification for RPC:
service = "service" serviceName "{" { option | rpc | emptyStatement } "}" rpc 
= "rpc" rpcName "(" [ "stream" ] messageType ")" "returns" "(" [ "stream" ] 
messageType 
")" (( "{" {option | emptyStatement } "}" ) | ";")

RPC refers to plain option. Does this mean that the following is valid and 
intended syntax?
rpc Foo (Req) returns (Res);
rpc Foo (Req) returns (Res) {option opt1 = "opt1"; option opt2 = true; };
rpc Foo (Req) returns (Res) {option opt1 = "opt1"; ;;; option opt2 = true; 
;;;};

I added number 3, because it seems like repetition is allowed together with 
emptyStatement. As a side note, I could not find any examples with RPC 
options as inspiration or to clear things up.

Further, I also find the specification for optionName a bit unclear.
Here's the specification for option and optionName:
option = "option" optionName "=" constant ";" optionName = ( ident | "(" 
["."] fullIdent ")" ) 
optionName does not seem to allow for repetition and fullIdent is only 
allowed within parenthesis(custom option). However multiple examples on 
https://protobuf.dev/programming-guides/proto3/#option-targets 
use custom-like options on the format:
option (bar).baz = "value";

For me it's pretty clear that the following optionNames are allowed:
foo
(foo)
(foo.bar)
(.foo)
(.foo.bar)

I don't really get which part of the specification that states that "baz", 
in the example above, is allowed outside of the parenthesis.

Thanks in advance,

/Mattias

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/protobuf/869b9637-ae04-440c-ade4-58874d4e3648n%40googlegroups.com.

[protobuf] Re: Documentation about endianness support

[Marc Gravell]

protobuf ensures that the *value of primitives* will be preserved; so if 
you send the uint32 23433451, it will be decoded as  23433451 regardless of 
the CPU/process endianness of sender and receiver. The details are 
here: https://protobuf.dev/programming-guides/encoding/ - but they **don't 
matter** unless you're writing your own protocol-level tooling.

What the receiver does with that value, however, is up to the receiver. For 
example, if they interpret that as IPv4 bytes using shift/mask operators: 
it will be work the same anywhere. If they interpret that as IPv4 bytes 
using "punning" or any other in-place re-interpret cast: then expect 
problems. But that's not an issue in the protobuf side - it is an issue in 
the code that *consumed it*.

On Saturday, 21 October 2023 at 07:48:03 UTC+1 Kumar bhat wrote:

> What happens if a uint32_t is sent from little endian system and received 
> by bit endian system. For eg: An Ipv4 address. Can we send the IP address 
> in any byte order and be able to see the same value in receiver of a 
> different endian machine? If thats the case, I dont see any documentation 
> for the same. 

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/protobuf/b5b87de0-6cd0-471a-b517-6d2e2f62f318n%40googlegroups.com.

Re: [protobuf] Documentation about endianness support

['Eric Salo' via Protocol Buffers]

An integer sent from one machine to another will preserve its value
regardless of the endianness of either machine. If you wish to attach
additional semantic meaning to that integer by using it to encode an ipv4
address then you need to take the appropriate steps to translate the local
value to and from network byte order, the same as you would for an int32
that was not part of a protobuf.

On Sat, Oct 21, 2023 at 1:48 AM Kumar bhat  wrote:

> What happens if a uint32_t is sent from little endian system and received
> by bit endian system. For eg: An Ipv4 address. Can we send the IP address
> in any byte order and be able to see the same value in receiver of a
> different endian machine? If thats the case, I dont see any documentation
> for the same.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to protobuf+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/protobuf/e9b8b750-3fe3-4f65-b5ab-4664f791568bn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/protobuf/CABV2wQXaZMhCKFQ-Srq%3DCbbYriH_M5u3j_YjYyjX94E98QHF0g%40mail.gmail.com.

[protobuf] Documentation about endianness support

[Kumar bhat]

What happens if a uint32_t is sent from little endian system and received 
by bit endian system. For eg: An Ipv4 address. Can we send the IP address 
in any byte order and be able to see the same value in receiver of a 
different endian machine? If thats the case, I dont see any documentation 
for the same. 

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/protobuf/e9b8b750-3fe3-4f65-b5ab-4664f791568bn%40googlegroups.com.