[protobuf] problem: protobuf (cpp) not serializing all fields to array?
Hi guys,
i am trying to serialize a message to an array and deserialize it on
the other end.
unforntunately it seems not everything gets serialized:
this is my proto file:
--
package protodm;
(...)
message AllConnections
{
required VC curVC = 1;
required VC oldVC = 2;
repeated Connection connections = 3;
}
message Connection
{
// required bytes curVC= 1;
required string partialConKey = 2;
repeated UIParam uiPars = 5;
repeated BOParam boPars = 6;
repeated STParam stParms= 7;
}
message VC
{
required uint32 runningNumber = 1;
required uint64 first = 2;
required uint64 second = 3;
}
--
and this is my code:
bool ProtoPackOwnConnections(boost::arraychar, 65535 buffer, int
usedBufLen)
{
protodm::AllConnections allCons;
OwnConMap_T::iterator it = myOwnConnections.begin();
char* curUnused = buffer.c_array();
while (it!=myOwnConnections.end())
{
boost::shared_ptrSP_Connection con = it-second;
protodm::Connection* c = allCons.add_connections();
con-ProtoPack(c);
++it;
}
::Mediornet::protodm::VC* theCurVC= allCons.mutable_curvc();
::Mediornet::protodm::VC* theOldVC= allCons.mutable_oldvc();
theCurVC-set_first(curVC.first);
theCurVC-set_second(curVC.second);
theCurVC-set_runningnumber(curVC.runningNumber);
theOldVC-set_first(oldVC.first);
theOldVC-set_second(oldVC.second);
theOldVC-set_runningnumber(oldVC.runningNumber);
mn_assert(allCons.has_curvc() allCons.oldvc().first() ==
oldVC.first, not set???);
usedBufLen = allCons.ByteSize();
google::protobuf::uint8* ptr =
allCons.SerializeWithCachedSizesToArray( (google::protobuf::uint8*)
buffer.c_array() );
std::string tmp = allCons.ShortDebugString();
DebugConsole(0, allCons as ShortDebugString: \n%s\n, tmp.c_str());
//testing unpack:
protodm::AllConnections unp;
if (!unp.ParseFromArray(buffer.c_array(), 0))
{
DebugConsole(0, Couldnt parse!);
return false;
}
return ptr!=NULL;
}
and this will be the output:
Firmware : allCons as ShortDebugString:
connections { partialConKey:
CO66_00_08__6A_03_057 uiPars { key { Keydef:
132 } value: 16777472 } uiPars { key { Keydef: 212 } value: 1 } uiPars
{ key { Keydef: 213 } value: 300 } uiPars { key { Keydef: 215 } value:
1279522249 } boPars { key { Keydef: 223 } value: false } boPars { key
{ Keydef: 224 } value: false } stParms { key { Keydef: 10 } value:
SW66.CO66_00_08__6A_03_057 }
stParms { key { Keydef: 226 } value: } stParms { key { Keydef:
16603 keyui: 0 } value: SW66.SL03.CH048 } stParms { key
{ Keydef: 16603 keyui: 1 } value: SW66.SL05.CH051 }
stParms { key { Keydef: 16603 keyui: 2 } value:
SW66.SL10.CH008 } stParms { key { Keydef: 16603 keyui:
3 } value: SW66.SL06.CH057 } stParms { key { Keydef:
16604 keyui: 0 } value: SW66.SL00.CH018 } stParms { key
{ Keydef: 16604 keyui: 1 } value: SW66.SL07.CH060 }
stParms { key { Keydef: 16604 keyui: 2 } value:
SW66.SL06.CH048 } stParms { key { Keydef: 16604 keyui:
3 } value: SW66.SL04.CH026 } } connections
{ partialConKey: CO66_00_20__39_06_060
uiPars { key { Keydef: 132 } value: 16777472 } uiPars { key { Keydef:
212 } value: 1 } uiPars { key { Keydef: 213 } value: 300 } uiPars
{ key { Keydef: 215 } value: 1279522249 } boPars { key { Keydef: 223 }
value: false } boPars { key { Keydef: 224 } value: false } stParms
{ key { Keydef: 10 } value:
SW66.CO66_00_20__39_06_060 }
stParms { key { Keydef: 226 } value: } stParms { key { Keydef:
16603 keyui: 0 } value: SW66.SL07.CH000 } stParms { key
{ Keydef: 16603 keyui: 1 } value: SW66.SL03.CH031 }
stParms { key { Keydef: 16603 keyui: 2 } value:
SW66.SL08.CH034 } stParms { key { Keydef: 16603 keyui:
3 } value: SW66.SL01.CH014 } stParms { key { Keydef:
16604 keyui: 0 } value: SW66.SL10.CH005 } stParms { key
{ Keydef: 16604 keyui: 1 } value: SW66.SL08.CH063 }
stParms { key { Keydef: 16604 keyui: 2 } value:
SW66.SL02.CH017 } stParms { key { Keydef: 16604 keyui:
3 } value: SW66.SL01.CH007 } } connections
{ partialConKey: CO66_00_20__78_10_062
uiPars { key { Keydef: 132 } value: 16777472 } uiPars { key {
[protobuf] Re: problem: protobuf (cpp) not serializing all fields to array?
d'ouh! you are right! (though the 0 was a copy-paste error from my code to the example code i posted i indeed had the wrong bytes length and usedBufLen was always 0) what a dumb mistake... thank you very much! On Jul 19, 2:27 pm, Oliver Jowett oliver.jow...@gmail.com wrote: phelyks wrote: if (!unp.ParseFromArray(buffer.c_array(), 0)) You are claiming that the array to parse from is zero bytes long. Unsurprisingly, trying to parse a zero-byte message doesn't give anything useful. Perhaps you meant to pass usedBufLen? -O -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Javas Message.writeDelimitedTo for C++ users
Hi, I'd like to make a suggestion: I think it would be quite convenient to have the writeDelimitedTo also available as a method of Message in C++. As far as I understand now in C++ I can archive the same with calling const int bufLen = 512; unsigned char buffer[bufLen]; google::protobuf::io::ArrayOutputStream arrayOutput(buffer, bufLength); google::protobuf::io::CodedOutputStream codedOutput(arrayOutput); Message protoMessage; codedOutput.WriteLittleEndian32(protoMessage.ByteSize()); protoMessage.SerializeToCodedStream(codedOutput); (though I still need to verify if its really the same) But of course, having a method doing that in vanilla protobuf would be a cool addition for C++ and I think it shouldnt be hard to implement or maintain? -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: Make check failed when cross compiling for power-pc
ok, just like i thought. thanks for the info On Mar 11, 8:35 pm, Kenton Varda ken...@google.com wrote: The tests obviously can't run when cross-compiling, so this is expected. You should manually run protobuf-test on the target architecture to make sure it works. The other tests probably aren't important. On Thu, Mar 11, 2010 at 2:12 AM, Felix Schmutz felix.schm...@gmail.comwrote: hi, i was trying to cross-compile protocol buffers 2.3.0 for power pc. when i did make-check, it failed and asked me to report it. i guess the make check just failed because i was doing the cross-compile and he then couldnt execute the cross-compiled executables. however when running configure, i explicitly told it --with-protoc=protoc so i thought the checks would work then? i am running on ubuntu 9.10 32bit and already have successfully built and installed protocol buffers on my machine. i am also able to successfully cross-compile apps for power-pc. here is what i did and the output of make check: protoc --version libprotoc 2.3.0 powerpc-linux-g++ --version .powerpc-glibc-linux-gnu-g++ (crosstool-NG-1.5.2) 4.3.4 Copyright (C) 2008 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. ./configure --with-protoc=protoc --target=powerpc-linux --host=powerpc-linux ... export PATH=$PATH:/opt/Rootfilesys/toolchain/ ... ./configure --with-protoc=protoc --target=powerpc-linux --host=powerpc-linux ... make clean ... make ... make check Making check in . make[1]: Entering directory `/home/felix/protobuf/protobuf-2.3.0' make check-local make[2]: Entering directory `/home/felix/protobuf/protobuf-2.3.0' Making lib/libgtest.a lib/libgtest_main.a in gtest make[3]: Entering directory `/home/felix/protobuf/protobuf-2.3.0/gtest' depbase=`echo src/gtest.lo | sed 's|[^/]*$|.deps/|;s|\.lo$||'`;\ /bin/bash ./libtool --tag=CXX --mode=compile powerpc-linux-g++ -DHAVE_CONFIG_H -I. -I./build-aux -I. -I./include -g -DNDEBUG -MT src/gtest.lo -MD -MP -MF $depbase.Tpo -c -o src/gtest.lo src/gtest.cc \ mv -f $depbase.Tpo $depbase.Plo libtool: compile: powerpc-linux-g++ -DHAVE_CONFIG_H -I. -I./build-aux -I. -I./include -g -DNDEBUG -MT src/gtest.lo -MD -MP -MF src/.deps/gtest.Tpo -c src/gtest.cc -fPIC -DPIC -o src/.libs/gtest.o libtool: compile: powerpc-linux-g++ -DHAVE_CONFIG_H -I. -I./build-aux -I. -I./include -g -DNDEBUG -MT src/gtest.lo -MD -MP -MF src/.deps/gtest.Tpo -c src/gtest.cc -o src/gtest.o /dev/null 21 depbase=`echo src/gtest-death-test.lo | sed 's|[^/]*$|.deps/|;s|\.lo$||'`;\ /bin/bash ./libtool --tag=CXX --mode=compile powerpc-linux-g++ -DHAVE_CONFIG_H -I. -I./build-aux -I. -I./include -g -DNDEBUG -MT src/gtest-death-test.lo -MD -MP -MF $depbase.Tpo -c -o src/gtest-death-test.lo src/gtest-death-test.cc \ mv -f $depbase.Tpo $depbase.Plo libtool: compile: powerpc-linux-g++ -DHAVE_CONFIG_H -I. -I./build-aux -I. -I./include -g -DNDEBUG -MT src/gtest-death-test.lo -MD -MP -MF src/.deps/gtest-death-test.Tpo -c src/gtest-death-test.cc -fPIC -DPIC -o src/.libs/gtest-death-test.o libtool: compile: powerpc-linux-g++ -DHAVE_CONFIG_H -I. -I./build-aux -I. -I./include -g -DNDEBUG -MT src/gtest-death-test.lo -MD -MP -MF src/.deps/gtest-death-test.Tpo -c src/gtest-death-test.cc -o src/gtest-death-test.o /dev/null 21 depbase=`echo src/gtest-filepath.lo | sed 's|[^/]*$|.deps/|;s|\.lo$||'`;\ /bin/bash ./libtool --tag=CXX --mode=compile powerpc-linux-g++ -DHAVE_CONFIG_H -I. -I./build-aux -I. -I./include -g -DNDEBUG -MT src/gtest-filepath.lo -MD -MP -MF $depbase.Tpo -c -o src/gtest-filepath.lo src/gtest-filepath.cc \ mv -f $depbase.Tpo $depbase.Plo libtool: compile: powerpc-linux-g++ -DHAVE_CONFIG_H -I. -I./build-aux -I. -I./include -g -DNDEBUG -MT src/gtest-filepath.lo -MD -MP -MF src/.deps/gtest-filepath.Tpo -c src/gtest-filepath.cc -fPIC -DPIC -o src/.libs/gtest-filepath.o libtool: compile: powerpc-linux-g++ -DHAVE_CONFIG_H -I. -I./build-aux -I. -I./include -g -DNDEBUG -MT src/gtest-filepath.lo -MD -MP -MF src/.deps/gtest-filepath.Tpo -c src/gtest-filepath.cc -o src/gtest-filepath.o /dev/null 21 depbase=`echo src/gtest-port.lo | sed 's|[^/]*$|.deps/|;s|\.lo$||'`;\ /bin/bash ./libtool --tag=CXX --mode=compile powerpc-linux-g++ -DHAVE_CONFIG_H -I. -I./build-aux -I. -I./include -g -DNDEBUG -MT src/gtest-port.lo -MD -MP -MF $depbase.Tpo -c -o src/gtest-port.lo src/gtest-port.cc \ mv -f $depbase.Tpo $depbase.Plo libtool: compile: powerpc-linux-g++ -DHAVE_CONFIG_H -I. -I./build-aux -I. -I./include -g -DNDEBUG -MT src/gtest-port.lo -MD -MP -MF src/.deps/gtest-port.Tpo -c src/gtest-port.cc -fPIC -DPIC -o src/.libs/gtest-port.o libtool: compile: powerpc-linux-g++ -DHAVE_CONFIG_H -I.
[protobuf] Re: protoc plugin compiler extension framework
ok, thanks for pointing me to the right directions and your RPC
system, that should be helpful.
On Feb 27, 12:45 am, Kenton Varda ken...@google.com wrote:
Plugins are now mentioned in the docs in several places:
http://code.google.com/apis/protocolbuffers/docs/reference/other.htmlhttp://code.google.com/apis/protocolbuffers/docs/reference/cpp-genera...
for java-generated and
python-generated)http://code.google.com/apis/protocolbuffers/docs/reference/cpp/google...http://code.google.com/apis/protocolbuffers/docs/reference/cpp/google...
I agree that some sort of tutorial would be nice but my tech writer is on
leave and as an engineer I suck at that sort of thing. :/
Here's a plugin I wrote for my own RPC system:
http://code.google.com/p/capnproto/source/browse/compiler/protoc-gen-...
On Fri, Feb 26, 2010 at 9:00 AM, phelyks felix.schm...@gmail.com wrote:
i think it would also be very helpful to have some sort of dummy-
example-plugin documentation.
at the moment i am completely lost how to begin to write the plugin.
but i dont want to whine: protobufs other documentation is excellent,
so maybe i am just getting too comfortable ;)
On Jan 6, 7:01 pm, Kenton Varda ken...@google.com wrote:
Yes. Sorry, I haven't had a chance to write up formal documentation yet.
See these two files:
http://code.google.com/p/protobuf/source/browse/trunk/src/google/prot
..
On Wed, Jan 6, 2010 at 1:29 AM, Chris hsifdr...@gmail.com wrote:
Is the plugin framework already part of 2.3.0? I can't find any
documentation for this new feature besides some early brainstorming
posts.
On Dec 22 2009, 7:28 pm, Kenton Varda ken...@google.com wrote:
The plugin framework is not meant for this. Plugins can only insert
code
at
points that have explicitly been declared by the original generator.
For
example, in Java, the code generator generates one insertion point in
each
class. So, you can add new methods to a message type, but you cannot
stick
javadoc comments on the existing methods.
I think that a system which let you arbitrarily edit the generated
code
would be too fragile -- any change to the code generator would
potentially
break plugins. In fact, I'm even worried that the current system is
risky
because it allows plugins to get access to private members which
could
change, but I don't see any way around that.
All this said, I think it would be great if the protocol compiler
supported
some format for documentation comments and automatically copied those
comments into the generated code. But no one has actually worked on
this
yet.
On Tue, Dec 22, 2009 at 6:42 AM, Christopher Piggott
cpigg...@gmail.com
wrote:
Hmm maybe I can use the UninterpretedOption message to do this.
Would something like this work?
message ChrisMessage {
option javadoc = This is an object representing Chris's Message;
repeated int32 field1 = 1 [javadoc=This is a javadoc for field
1];
repeated int32 field2 = 2 [javadoc=This is a javadoc for field
2];
}
Then write a plug-in that finds those and writes the ones whose
NamePart.equals(javadoc) in as a /** comment */
Possible?
--
You received this message because you are subscribed to the Google
Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com
protobuf%2bunsubscr...@googlegroups.comprotobuf%252bunsubscr...@googlegroups.com
protobuf%2bunsubscr...@googlegroups.comprotobuf%252bunsubscr...@googlegroups.com
protobuf%252bunsubscr...@googlegroups.comprotobuf%25252bunsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
--
You received this message because you are subscribed to the Google
Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com
protobuf%2bunsubscr...@googlegroups.comprotobuf%252bunsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
--
You received this message because you are subscribed to the Google Groups
Protocol
[protobuf] Re: protoc plugin compiler extension framework
i think it would also be very helpful to have some sort of dummy-
example-plugin documentation.
at the moment i am completely lost how to begin to write the plugin.
but i dont want to whine: protobufs other documentation is excellent,
so maybe i am just getting too comfortable ;)
On Jan 6, 7:01 pm, Kenton Varda ken...@google.com wrote:
Yes. Sorry, I haven't had a chance to write up formal documentation yet.
See these two files:
http://code.google.com/p/protobuf/source/browse/trunk/src/google/prot...http://code.google.com/p/protobuf/source/browse/trunk/src/google/prot...
On Wed, Jan 6, 2010 at 1:29 AM, Chris hsifdr...@gmail.com wrote:
Is the plugin framework already part of 2.3.0? I can't find any
documentation for this new feature besides some early brainstorming
posts.
On Dec 22 2009, 7:28 pm, Kenton Varda ken...@google.com wrote:
The plugin framework is not meant for this. Plugins can only insert code
at
points that have explicitly been declared by the original generator. For
example, in Java, the code generator generates one insertion point in
each
class. So, you can add new methods to a message type, but you cannot
stick
javadoc comments on the existing methods.
I think that a system which let you arbitrarily edit the generated code
would be too fragile -- any change to the code generator would
potentially
break plugins. In fact, I'm even worried that the current system is
risky
because it allows plugins to get access to private members which could
change, but I don't see any way around that.
All this said, I think it would be great if the protocol compiler
supported
some format for documentation comments and automatically copied those
comments into the generated code. But no one has actually worked on this
yet.
On Tue, Dec 22, 2009 at 6:42 AM, Christopher Piggott cpigg...@gmail.com
wrote:
Hmm maybe I can use the UninterpretedOption message to do this.
Would something like this work?
message ChrisMessage {
option javadoc = This is an object representing Chris's Message;
repeated int32 field1 = 1 [javadoc=This is a javadoc for field 1];
repeated int32 field2 = 2 [javadoc=This is a javadoc for field 2];
}
Then write a plug-in that finds those and writes the ones whose
NamePart.equals(javadoc) in as a /** comment */
Possible?
--
You received this message because you are subscribed to the Google
Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com
protobuf%2bunsubscr...@googlegroups.comprotobuf%252bunsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
[protobuf] how to define a one-way rpc message?
Hi all!
I am currently playing around with protocol buffers and now was
looking closer at the rpc code that can be generated by it.
But I am a bit confused. I did not find a way how to create rpc
messages that do not define a reply.
is there a way to create one-way rpc calls in protocol buffers?
An example what i would like would be that:
service myService
{
rpc Echo (EchoParam) returns (EchoReply) ;
rpc OneWayWithoutResponse(ParamsToSend);
}
the only workaround that i can think of would be creating a Void
message that is declared as the answer and the rpc implementation to
then ignore those.
like this:
message Void
{
}
service myService
{
rpc OneWayWithoutResponse(FirmwareEvent) returns (Void);
}
is there a way how i can avoid this workaround?
cheers phelyks
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
