[protobuf] Setting string fields
Hello,
I'm having a strange problem when setting an string value in a protocol
buffer. Keep in mind that my experience in C++ and PB is rather limited.
The .proto file looks more or less like this:
--
package buffers;
message AttributeValue {
enum Type {
INT64 = 0;
UINT64 = 1;
INT32 = 2;
UINT32 = 3;
FLOAT = 4;
DOUBLE = 5;
INVALID = 6;
STRING = 7;
}
required Type type = 1;
optional int32 i32 = 2;
optional int64 i64 = 3;
optional uint32 ui32 = 4;
optional uint64 ui64 = 5;
optional float f = 6;
optionaldouble d = 7;
optional string str = 8;
}
--
I also have a function to convert a class VariableDatatype into an
AttributeValue buffer. It looks like this:
void BufferConverter::attribute_value(const VariableDatatype s_atv,
buffers::AttributeValue b_atv)
{
switch(s_atv.type()) {
case VariableDatatype::Type::STRING:
b_atv.set_type(buffers::AttributeValue::STRING);
b_atv.set_str(s_atv.str());
break;
}
}
In the main function I try to create a protocol buffer containing the
string stored in VariableDatatype by using the attribute_value() function
in the way shown below:
main-1()
{
buffers::AttributeValue *bs = new buffers::AttributeValue();
c.attribute_value(ds, *bs);
}
This doesn't work.
If I print the value of b_atv inside the attribute_value function using gdb
I get something like:
gdb print b_atv
26 = (buffers::AttributeValue ) @0x6496b0: {google::protobuf::Message =
{google::protobuf::MessageLite = {
However, when I print *bs from inside main-1, I get something different:
gdb print *bs
$36 = {google::protobuf::Message = {google::protobuf::MessageLite =
{_vptr.MessageLite = 0x642650 vtable for buffers::AttributeValue+16
I can work around the problem by using the following main-2:
main-2()
{
buffers::AttributeValue *bs = new buffers::AttributeValue();
buffers::AttributeValue bss = *bs;
c.attribute_value(ds, bss);
}
but that's not very elegant. Can someone point me to the right way to
handle this?
Thank you
A.
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/groups/opt_out.
[protobuf] Problem parsing extended data dynamically
I have exteded data in my .proto file:
option java_outer_classname = Foobar;
message Foo {
optional int32 i = 1;
extensions 10 to 9;
}
message Bar {
extend Foo {
optional int32 j = 10001;
optional string name = 10002;
}
}
message Msg {
optional Foo foo = 1;
}
So, here is code which i'm using to parse data using DynamicMessage:
// Create data
Foobar.Foo foo = Foobar.Foo.newBuilder()
.setI(123)
.setExtension(Foobar.Bar.j, 456)
.setExtension(Foobar.Bar.name, simpleName)
.build();
Foobar.Msg msg = Foobar.Msg.newBuilder().setFoo(foo).build();
byte[] msgData = msg.toByteArray();
// register extensions
ExtensionRegistry registry = ExtensionRegistry.newInstance();
Foobar.registerAllExtensions(registry);
final DescriptorProtos.FileDescriptorSet fds =
DescriptorProtos.FileDescriptorSet.parseFrom(new
FileInputStream(foobar.desc));
Descriptors.Descriptor descriptor =
Descriptors.FileDescriptor.buildFrom(fds.getFile(0), new
Descriptors.FileDescriptor[]{})
.findMessageTypeByName(Msg);
// build Msg from data using dynamically created descriptor
DynamicMessage dynamicMessage =
DynamicMessage.parseFrom(descriptor, msgData, registry);
System.out.println(dynamically created descriptor: +
JsonFormat.printToString(dynamicMessage));
// build Msg from data using manually created descriptor
DynamicMessage dynamicMessage1 =
DynamicMessage.parseFrom(Foobar.Msg.getDescriptor(), msgData, registry);
System.out.println(manually created descriptor: +
JsonFormat.printToString(dynamicMessage1));
Result:
dynamically created descriptor: {foo: {i: 123, 10001: [456], 10002:
[simpleName]}}
manually created descriptor: {foo: {i: 123,Bar.j: 456,Bar.name:
simpleName}}
So, dynamicMessage1 parsed correctly - it has fields Bar.j and Bar.name
which were parsed as extensions.
But dynamicMessage parsed incorrectly - those fields parsed as unknown
fields and their names are equal to thier numbers - 10001 and 10002.
May be problem is in getting descriptor for the Msg message? Is it
correct way to get the descriptor dynamicaly?
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [protobuf] Setting string fields
The problem is that if I use main-1 the string value is lost when leaving
the scope of the attribute_value() function.
Am I doing something wrong?
On Monday, August 19, 2013 7:17:56 PM UTC+2, Ilia Mirkin wrote:
Perhaps I'm dense, but what's the problem? That gdb doesn't print the
exact type info that you expected? That could well be the difference
between -O2 and -O0...
On Mon, Aug 19, 2013 at 9:55 AM, Alvaro Aguilera
alvaro@gmail.com javascript: wrote:
Hello,
I'm having a strange problem when setting an string value in a protocol
buffer. Keep in mind that my experience in C++ and PB is rather limited.
The .proto file looks more or less like this:
--
package buffers;
message AttributeValue {
enum Type {
INT64 = 0;
UINT64 = 1;
INT32 = 2;
UINT32 = 3;
FLOAT = 4;
DOUBLE = 5;
INVALID = 6;
STRING = 7;
}
required Type type = 1;
optional int32 i32 = 2;
optional int64 i64 = 3;
optional uint32 ui32 = 4;
optional uint64 ui64 = 5;
optional float f = 6;
optionaldouble d = 7;
optional string str = 8;
}
--
I also have a function to convert a class VariableDatatype into an
AttributeValue buffer. It looks like this:
void BufferConverter::attribute_value(const VariableDatatype s_atv,
buffers::AttributeValue b_atv)
{
switch(s_atv.type()) {
case VariableDatatype::Type::STRING:
b_atv.set_type(buffers::AttributeValue::STRING);
b_atv.set_str(s_atv.str());
break;
}
}
In the main function I try to create a protocol buffer containing the
string
stored in VariableDatatype by using the attribute_value() function in
the
way shown below:
main-1()
{
buffers::AttributeValue *bs = new buffers::AttributeValue();
c.attribute_value(ds, *bs);
}
This doesn't work.
If I print the value of b_atv inside the attribute_value function using
gdb
I get something like:
gdb print b_atv
26 = (buffers::AttributeValue ) @0x6496b0: {google::protobuf::Message
=
{google::protobuf::MessageLite = {
However, when I print *bs from inside main-1, I get something different:
gdb print *bs
$36 = {google::protobuf::Message = {google::protobuf::MessageLite =
{_vptr.MessageLite = 0x642650 vtable for buffers::AttributeValue+16
I can work around the problem by using the following main-2:
main-2()
{
buffers::AttributeValue *bs = new buffers::AttributeValue();
buffers::AttributeValue bss = *bs;
c.attribute_value(ds, bss);
}
but that's not very elegant. Can someone point me to the right way to
handle
this?
Thank you
A.
--
You received this message because you are subscribed to the Google
Groups
Protocol Buffers group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to protobuf+u...@googlegroups.com javascript:.
To post to this group, send email to prot...@googlegroups.comjavascript:.
Visit this group at http://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/groups/opt_out.
[protobuf] Uninstall prtobuff
Hi all, I am having two versions of protobuff ( 2.4.1 and 2.5.0) i want to remove one off them. Can any one help me with that? -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at http://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/groups/opt_out.
Re: [protobuf] deserializing unknown protobuf message?
Something like protoc --decode_raw?
As for manipulating these in code, there's nothing too great available
if you don't have a descriptor at all. You could create a dummy
message (no fields) and then get them via unknown fields in the
message's reflection object, but they're not easy to manipulate. Note
that the encoded information is not always sufficient for correct
decoding. For example, strings, submessages, and perhaps packed arrays
are encoded in the same way. I also don't think that whether zigzag
encoding is used on varints is part of the serialized data. You really
really really want a descriptor. If not compiled in, then passed along
with the data.
-ilia
On Mon, Aug 19, 2013 at 3:37 PM, Tom Ward tw...@thefoundry.co.uk wrote:
Hi all,
I'm just getting to grips with protobuf and, having read through the
documentation, I'm struggling to find what I'm after.
Basically I'm trying to work out how to deserialize a protobuf message
without using the generated headers, as we're likely to get messages that
weren't generated at compile time. I've looked through the documentation,
but I only seem to be able to find ones that use generated classes to
deserialize, or that use a Descriptor from a generated class to create a
DynamicMessage, which I can't seem to work out how to do when we don't have
the proto.
Here's a very simple example of what I mean, where Message* is set to some
base type that allows deserialization, and then can use the reflection API
(or some factory) to correctly deal with the message:
#include sstream
#include google/protobuf/message.h
int main(int argc, char **argv)
{
if( argc 1)
{
// This will fail to compile, as Message is pure abstract
google::protobuf::Message* message = NULL; //new
google::protobuf::Message();
std::istringstream ss(std::string(argv[1],strlen(argv[1])));
message-ParseFromIstream(ss);
// do something with the message
// ...
}
return 0;
}
Apologies in advance if this has already been answered...
Thanks
Tom
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To unsubscribe from this group and stop receiving emails from it, send an
email to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [protobuf] deserializing unknown protobuf message?
On Mon, Aug 19, 2013 at 8:37 PM, Tom Ward tw...@thefoundry.co.uk wrote: Basically I'm trying to work out how to deserialize a protobuf message without using the generated headers, as we're likely to get messages that weren't generated at compile time. I've looked through the documentation, but I only seem to be able to find ones that use generated classes to deserialize, or that use a Descriptor from a generated class to create a DynamicMessage, which I can't seem to work out how to do when we don't have the proto. The protobuf encoding isn't self-describing; generally you need either a compiled message definition or a Descriptor to make sense of an encoded message. Without one of those, there is not much you can do beyond splitting the message into opaque fields that you will have difficulty in interpreting further. As Descriptors can be turned into protobuf messages, one approach is to include a serialized Descriptor as part of the standard message framing; then you can use DynamicMessage on the receiver side. See https://developers.google.com/protocol-buffers/docs/techniques#self-description Oliver -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at http://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/groups/opt_out.
[protobuf] ~FileDescriptorTables receive SIGABRT
I am using protobuf for communication between clients and servers. When I use valgrind to check the client binary, the following messages are listed. It seems there is some possible lost memory in __static_initialization_and_destruction_0. But why does this appear? How to suppress it? ==2928== 43 bytes in 1 blocks are possibly lost in loss record 549 of 711 ==2928==at 0x4C2B1C7: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==2928==by 0x5A0EA88: std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocatorchar const) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.16) ==2928==by 0x5A0F7FA: std::string::_Rep::_M_clone(std::allocatorchar const, unsigned long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.16) ==2928==by 0x5A0FF5B: std::basic_stringchar, std::char_traitschar, std::allocatorchar ::basic_string(std::string const) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.16) ==2928==by 0x529E800: google::protobuf::SimpleDescriptorDatabase::DescriptorIndexstd::pairvoid const*, int ::AddFile(google::protobuf::FileDescriptorProto const, std::pairvoid const*, int) (in /usr/lib/libprotobuf.so.7.0.0) ==2928==by 0x529A677: google::protobuf::EncodedDescriptorDatabase::Add(void const*, int) (in /usr/lib/libprotobuf.so.7.0.0) ==2928==by 0x52677A6: google::protobuf::DescriptorPool::InternalAddGeneratedFile(void const*, int) (in /usr/lib/libprotobuf.so.7.0.0) ==2928==by 0x427CB0: com::ots::protobuf_AddDesc_ots_5fprotocol_2eproto() (ots_protocol.pb.cc:1075) ==2928==by 0x4507FE: com::ots::StaticDescriptorInitializer_ots_5fprotocol_2eproto::StaticDescriptorInitializer_ots_5fprotocol_2eproto() (ots_protocol.pb.cc:1162) ==2928==by 0x44D43D: __static_initialization_and_destruction_0(int, int) (ots_protocol.pb.cc:1164) ==2928==by 0x44D452: _GLOBAL__sub_I__ZN3com3ots42protobuf_AssignDesc_ots_5fprotocol_2eprotoEv (ots_protocol.pb.cc:11800) ==2928==by 0x45C66C: __libc_csu_init (in /home/ots/unittest/test_ots_client) In normal cases, the client runs very well. However, when I increase threads to 50, the program receives SIGABRT in __do_global_dtors_aux. The backtrace is listed below. #0 0x00354ea30265 in raise () from /lib64/libc.so.6 #1 0x00354ea31d10 in abort () from /lib64/libc.so.6 #2 0x00354ea6a84b in __libc_message () from /lib64/libc.so.6 #3 0x00354ea72856 in free () from /lib64/libc.so.6 #4 0x01e433cf in clear (this=0x32a7760) at /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/ext/new_allocator.h:94 #5 0x01e46668 in ~FileDescriptorTables (this=0x32a76c0) at /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/ext/hashtable.h:361 #6 0x00354ea3368e in __cxa_finalize () from /lib64/libc.so.6 #7 0x2b47191796a6 in __do_global_dtors_aux () from /home/ots/build/debug64/lib/libprotobuf.so.7 #8 0x in ?? () The message from valgrind and the SIGABRT error indicate that the root cause is the Descriptor. But the initialization and destruction of Descriptors are done by libprotobuf.so automatically, and I modify nothing about them. Anyone has any idea? -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at http://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/groups/opt_out.
