[protobuf] Self-describing messages in Python
Hello,
I am trying to build self-describing messages using Python. I read through
the
documentation
https://protobuf.dev/programming-guides/techniques/#self-description
as well as through the Python
API https://googleapis.dev/python/protobuf/latest/google/protobuf.html, but
I am still not very sure about how to do that.
I started with the example message
syntax = "proto3"; import "google/protobuf/any.proto"; import
"google/protobuf/descriptor.proto"; message SelfDescribingMessage { // Set
of FileDescriptorProtos which describe the type and its dependencies.
google.protobuf.FileDescriptorSet descriptor_set = 1; // The message and
its type, encoded as an Any message. google.protobuf.Any message = 2; }
And I would like to pass through it a simple addressbook message
syntax = "proto2";
package tutorial;
message AddressBook {
optional string name = 1;
optional string number = 2;
}
The AddressBook message would be contained in the message field and the
descriptor_set field would contain the descriptor of the proto above.
However I could not do this no matter what I have tried.
The closest I got was exporting a file descriptor set from the
SelfDescribingMessage.proto given in the documentation as "protoc
--proto_path=. --proto_path=./include
--descriptor_set_out=./self_describing_ds --include_imports
self_describing.proto" and then reading it
with open("self_describing_ds", 'rb') as fh:
fds = descriptor_pb2.FileDescriptorSet.FromString(fh.read())
message_classes = message_factory.GetMessages(fds.file)
my_proto_instance = message_classes["SelfDescribingMessage"]()
address_book = addressbook_pb2.AddressBook()
address_book.name = "John Doe"
address_book.number = "123456"
my_proto_instance.message.Pack(address_book)
I am not able to set my_proto_instance.descriptor_set though. Extracting
address book descriptor set using protoc and then reading it and trying to
append it
with open("addressbook_ds", 'rb') as fh:
addressbook_fds =
descriptor_pb2.FileDescriptorSet.FromString(fh.read())
my_proto_instance.descriptor_set.file.append(addressbook_fds)
fails on
TypeError: Parameter to MergeFrom() must be instance of same class:
expected got .
I could not get any closer.
Does anyone have any simple example on sending the self-describing messages
in Python, please?
Thanks a lot in advance.
Jan
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/protobuf/33d19ba2-d008-49d6-b54b-2075113cb5e4n%40googlegroups.com.
Re: [protobuf] Self describing messages
On Wed, Jun 22, 2011 at 3:30 AM, slookin wrote:
> I trying to develop "flexible" server for reciving message, i'm not
> able to create java classes for each message type, but I can upload
> proto descripter on server. Of course client (sender) will use
> generated java classes for prepare messages:
>
> My code (sender):
> Person.Builder person = Person.newBuilder();
> person.setId(Integer.valueOf(42));
> person.setEmail("test_em...@gmail.com");
> person.setName("Viktor Villari");
> Person p=person.build();
> FileOutputStream fstream = new FileOutputStream ("message.pf");
> CodedOutputStream outSream = CodedOutputStream.newInstance(fstream);
> p.writeTo(outSream);
> outSream.flush();
> System.out.println("sent");
>
> Reciver (address.descriptor.proto - grenerated via descriptor.proto):
>FileInputStream input = new FileInputStream
> ("address.descriptor.proto");
>DescriptorProtos.FileDescriptorSet
> fdsProto=DescriptorProtos.FileDescriptorSet.parseFrom(input);
>input = new FileInputStream ("message.pf");
>//
> System.out.println(fds.getFile(0).getMessageType(0).getName());
>System.out.println("File name =
> "+fdsProto.getFile(0).getName());
>// System.out.println("Message type =
> "+fdsProto.getFile(0).getMessageType(0).getName());
>// System.out.println("Field info =
> "+fdsProto.getFile(0).getMessageType(0).getField(0).getName()+"
> "+fdsProto.getFile(0).getMessageType(0).getField(0).getType());
>FileDescriptor
> fileDescr=FileDescriptor.buildFrom(fdsProto.getFile(0), new
> FileDescriptor[0]);
>System.out.println("Message type =
> "+fileDescr.getMessageTypes().get(0).getName());
>System.out.println("Field info =
> "+fileDescr.getMessageTypes().get(0).getFields().get(0).getName()+"
> type=
> "+fileDescr.getMessageTypes().get(0).getFields().get(0).getType());
>DynamicMessage
> dm=DynamicMessage.parseFrom(fdsProto.getDescriptor(),
> input);
>
On this line, you are passing the descriptor for the FileDescriptorSet. You
should be using fileDescr to get at the Person message type. (Note that your
description scheme does not indicate what message type is actually used. But
I also don't see the code snippet that serializes the FileDescriptorSet, or
for that matter delimit between the FileDescriptorSet and the serialized
Person data, so maybe you are doing something elsewhere)
>System.out.println("DynamicMessage to string
> \n"+dm.toString());
>// problem line
>System.out.println("Person.name = "+
> dm.getField(fileDescr.getMessageTypes().get(0).getFields().get(0)));
>//
>
>
> Output:
> File name = addressbook.proto
> Message type = Person
> Field info = name type= STRING
> DynamicMessage to string
> 2: "Viktor Villari"
> 3: 42
> 4: "test_em...@gmail.com"
>
> Exception in thread "main" java.lang.IllegalArgumentException:
> FieldDescriptor does not match message type.
>at
>
> com.google.protobuf.DynamicMessage.verifyContainingType(DynamicMessage.java:
> 242)
>at com.google.protobuf.DynamicMessage.getField(DynamicMessage.java:
> 160)
>at net.lookin.protobuf.Test.main(Test.java:64)
>
> Why I recive exception?
> How I can access to specific field value in DynamicMessage?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To post to this group, send email to protobuf@googlegroups.com.
> To unsubscribe from this group, send email to
> protobuf+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/protobuf?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
[protobuf] Self describing messages
I trying to develop "flexible" server for reciving message, i'm not
able to create java classes for each message type, but I can upload
proto descripter on server. Of course client (sender) will use
generated java classes for prepare messages:
My code (sender):
Person.Builder person = Person.newBuilder();
person.setId(Integer.valueOf(42));
person.setEmail("test_em...@gmail.com");
person.setName("Viktor Villari");
Person p=person.build();
FileOutputStream fstream = new FileOutputStream ("message.pf");
CodedOutputStream outSream = CodedOutputStream.newInstance(fstream);
p.writeTo(outSream);
outSream.flush();
System.out.println("sent");
Reciver (address.descriptor.proto - grenerated via descriptor.proto):
FileInputStream input = new FileInputStream
("address.descriptor.proto");
DescriptorProtos.FileDescriptorSet
fdsProto=DescriptorProtos.FileDescriptorSet.parseFrom(input);
input = new FileInputStream ("message.pf");
//
System.out.println(fds.getFile(0).getMessageType(0).getName());
System.out.println("File name =
"+fdsProto.getFile(0).getName());
// System.out.println("Message type =
"+fdsProto.getFile(0).getMessageType(0).getName());
// System.out.println("Field info =
"+fdsProto.getFile(0).getMessageType(0).getField(0).getName()+"
"+fdsProto.getFile(0).getMessageType(0).getField(0).getType());
FileDescriptor
fileDescr=FileDescriptor.buildFrom(fdsProto.getFile(0), new
FileDescriptor[0]);
System.out.println("Message type =
"+fileDescr.getMessageTypes().get(0).getName());
System.out.println("Field info =
"+fileDescr.getMessageTypes().get(0).getFields().get(0).getName()+"
type=
"+fileDescr.getMessageTypes().get(0).getFields().get(0).getType());
DynamicMessage
dm=DynamicMessage.parseFrom(fdsProto.getDescriptor(),
input);
System.out.println("DynamicMessage to string \n"+dm.toString());
// problem line
System.out.println("Person.name = "+
dm.getField(fileDescr.getMessageTypes().get(0).getFields().get(0)));
//
Output:
File name = addressbook.proto
Message type = Person
Field info = name type= STRING
DynamicMessage to string
2: "Viktor Villari"
3: 42
4: "test_em...@gmail.com"
Exception in thread "main" java.lang.IllegalArgumentException:
FieldDescriptor does not match message type.
at
com.google.protobuf.DynamicMessage.verifyContainingType(DynamicMessage.java:
242)
at com.google.protobuf.DynamicMessage.getField(DynamicMessage.java:
160)
at net.lookin.protobuf.Test.main(Test.java:64)
Why I recive exception?
How I can access to specific field value in DynamicMessage?
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Self describing messages
You probably need a FileDescriptorSet, which has the whole enchilada. --Chris On Jul 23, 2010, at 12:08 PM, David wrote: > I'm trying to send a self-describing message from a JMS producer to a > consumer. > > On the producer I have something like: > >Descriptor desc = builder.getDescriptorForType(); >DescriptorProto proto = desc.toProto(); > > Then I send the bytes of the proto and the bytes of the built object > (from builder). > > On the consumer side, I can't figure out how to take the > DescriptorProto I get on the wire and make a Descriptor out of it > which I can use to parse the bytes of the object using > DynamicMessage. I've seen one mention of using FileDescriptorProto, > but I'm not working with those... just DescriptorProto. > > Can someone point me in the right direction? > > Thanks, > -David- > > -- > You received this message because you are subscribed to the Google Groups > "Protocol Buffers" group. > To post to this group, send email to proto...@googlegroups.com. > To unsubscribe from this group, send email to > protobuf+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/protobuf?hl=en. > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Self describing messages
I'm trying to send a self-describing message from a JMS producer to a consumer. On the producer I have something like: Descriptor desc = builder.getDescriptorForType(); DescriptorProto proto = desc.toProto(); Then I send the bytes of the proto and the bytes of the built object (from builder). On the consumer side, I can't figure out how to take the DescriptorProto I get on the wire and make a Descriptor out of it which I can use to parse the bytes of the object using DynamicMessage. I've seen one mention of using FileDescriptorProto, but I'm not working with those... just DescriptorProto. Can someone point me in the right direction? Thanks, -David- -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
