Re: Deserializing Messages of unknown type at compile-time
Even if the message contains only one, non-repeated field, ParseFrom*() will
keep reading until EOF or an error.
At Google, we have lots of various container formats, for streaming,
record-based files, database tables, etc., where each record is a protocol
buffer. All of these formats store the size of the message before the
message itself. Our philosophy is that because we have protocol buffers,
all of these *other* formats and protocols can be designed to pass around
arbitrary byte blobs, which greatly simplifies them. An arbitrary byte blob
is not necessarily self-delimiting, so it's up to these container formats to
keep track of the size separately.
On Thu, Sep 11, 2008 at 8:20 PM, [EMAIL PROTECTED] wrote:
Kenton,
No, it won't work. Protocol buffers are not self-delimiting. They
assume
that the input you provide is supposed to be one complete message, not a
message possibly followed by other stuff.
There are a couple of related threads about delimiting the outer
message (with either a marker or a length). The need for this seems to
arise from streaming (especially when input would block such as on a
network socket).
Could this not be solved by a simple convention in the proto file ?
(Maybe I am missing something big here)
Let us say we have a proto as follows
message TRPProtocol
{
message TRPPDU
{
required int32 version;
required int32 type;
optional HelloRequest hello_req = 1;
optional HelloResponse hello_resp = 2;
optional ConnectRequest connect_req =
3;
// etc etc
};
required TRPPDU thepdu=1;
};
On the wire the outer message is not length delimited, but the inner
message is. The inner message is represented by the 'required' field
'thepdu'.
It would then be possible to stream instances of the inner message
TRPPDU. I hope my understanding is correct. Could you write
something like the following ?
TRPProtocol::TRPPDU Pdu;
Pdu.ParseFromFileDescriptor( socket_fd); // socket_fd has been
opened and initialized earlier
to read just one message, respond to that if needed, and then read the
next one.
Is my understanding correct ? Is this how it is done at Google when
using PB for client - server comms ?
Thanks,
Vivek
You will need to somehow communicate the size of the message and make
sure
to limit the input to that size.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: How to properly implement RpcChannel::CallMethod
You should send the method's name along with the request. The receiver can
then look up the MethodDescriptor from the service's ServiceDescriptor,
using FindMethodByName().
You might even encode your messages like this:
message RpcRequest {
required string method_name = 1;
// The serialized request message.
required bytes request_data = 2;
}
On Tue, Sep 16, 2008 at 8:54 AM, [EMAIL PROTECTED] wrote:
Hi!
I have some problems setting up a simple IPC system using protocol
buffers for sending the parameters and results between two processes.
I have written a protocol buffers service with a function and a simple
IPC system that can transfer raw byte messages between two processes.
My problem is understanding how my RpcChannel::CallMethod should be
implemented. I was expecting it was as easy to encode the message to
raw bytes and pick it up in the other process and decode it. However,
it turned out that the MethodDescriptor is not encoded in the Message
and it's needed in the other end to route the message to the right
function. I found no functionality to serialize the MethodDescriptor
to a byte stream.
Of course I can come up with encodings for the method descriptor,
either sending the name or the index should work. Then I can just put
it first in the buffer that's transferred to the other process, but
that means I lose the nice feature of letting protocol buffers
handling unicode string encoding, endianness of integers etc. One
approach would be to create a meta message containing both the method
descriptor and the encoded message as a variable size byte array in
one message.
I somehow think I'm missing the point here, how is
RpcChannel::CallMethod supposed to work?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: Protocol Buffers can't use in visual studio 2003 ?
I don't think DLLs or LIBs created with a never version of MSVC will work with MSVC 2003. Protobufs use STL heavily in the interface, and I think the STL classes change from version to version. On Tue, Sep 16, 2008 at 4:44 AM, Niall [EMAIL PROTECTED] wrote: I get the same runtime error with MSVC 03. Could some make available the required *.dlls and *.libs which I could download and test with my IDE? This should tell whether it's the dll/ libs that MSVC 2003 makes are the root of the problem or whether it's with program that's linking to them? Right? you can create a new project under the solution of vsprojects, then select the dependencies to lib*, then it will work. I'm not certain what you mean there... Thanks, Niall --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Protocol Buffers can't use in visual studio 2003 ?
Even if only the implementation changes -- specifically, the class's memory layout -- that will cause a binary incompatibility, especially since STL is all templates. It seems the debug vs. release versions of MSVC's STL are not even binary compatible, so being compatible between two versions of MSVC seems doubtful. On Tue, Sep 16, 2008 at 10:40 AM, Niall [EMAIL PROTECTED] wrote: Thanks for the reply. You're probably right... but I'm still not convinced it wouldn't work. I couldn't imagine the STL's interface changing. The implementation, sure, but I'd be surprised if the interface would (There'd need to be a real good reason with all that reference paperwork needing changing). Protobufs use STL heavily in the interface, and I think the STL classes change from version to version. Sure, but if I can compile the DLL's and LIB's anyway I can't see this being a problem. This is just gut feeling, though. I can't find anything online to say either way which would happen. Would it be worth trying anyway? If possible. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Protocol Buffers can't use in visual studio 2003 ?
I think you'll have to run it in a debugger and try to figure out the cause of the crash you're seeing. I don't have a copy of MSVC '03, but if you send me the stack trace I could try to guess what's going on. On Tue, Sep 16, 2008 at 1:03 PM, Niall [EMAIL PROTECTED] wrote: Finally. I see what you mean. Thanks for your help and clarification. I wasn't aware of those issues. Thanks again! Any suggestions on how I could use protobufs with MSVC '03? Without the runtime errors when calling SerializeToOstream, that is. Lenjoy, in the second post, said: you can create a new project under the solution of vsprojects, then select the dependencies to lib*, then it will work. But I don't know what that means... I tried a couple of things, like adding a new project to the solution I used in generating the DLLs, libs etc. But with the same result. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Emulate hash_map using map when necessary.
submitted. Thanks for prodding me into fixing this. :) On Fri, Sep 19, 2008 at 6:41 AM, [EMAIL PROTECTED] wrote: You can submit it, the code is correct. Thank you. -- *De :* Kenton Varda [mailto:[EMAIL PROTECTED] *Envoyé :* 18 septembre 2008 17:13 *À :* [EMAIL PROTECTED]; Choinière, Vincent; protobuf@googlegroups.com *Objet :* Re: Emulate hash_map using map when necessary. Any comments on the code? If not, I'll go ahead and submit it. On Thu, Sep 18, 2008 at 1:59 PM, [EMAIL PROTECTED] wrote: Yes, is working. == Test ok on Tru64 (with some other changed in the code for the compiler) == Test ok on Windows Visual Studio 2005 Issue 5682 can be ignored... http://codereview.appspot.com/5683 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Any small sample Class Service code for Java
On Fri, Sep 19, 2008 at 9:50 AM, Mars [EMAIL PROTECTED] wrote:
just want to find some sample code for Class Service:
http://code.google.com/apis/protocolbuffers/docs/reference/python/google.protobuf.service.Service-class.html
That's the Python version. Shouldn't you be looking at the Java version?
http://code.google.com/apis/protocolbuffers/docs/reference/java/com/google/protobuf/Service.html
for example:
given .proto .java files below,
how to use simply use them in some basic Java network program, such as
ServerSocket or DatagramSocket?
Maybe look at some of the RPC implementations people are working on:
http://code.google.com/p/protobuf/wiki/RPCImplementations
=
# .proto
message Req1 {
required int32 id = 1;
}
message Res1 {
required string name = 1;
}
service Serv1 {
rpc Func1(Req1) returns (Res1) {
option name = name1;
}
}
==
// .java
public class MyServ implements Serv1 {
public EmptyMessage Func1(RPC rpc, Req1 req) throws RpcException {
//...
}
}
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: Does SerializeToXxx() guarantee a call to ByteSize()?
You probably shouldn't rely on this. Some of the serialization methods will guarantee this, some won't. What you can do is call ByteSize() yourself, and then use SerializeWithCachedSizes(), which requires that ByteSize() was already called and won't call it again. You'll have to do a small amount of extra setup (allocating a ZeroCopyOutputStream and a CodedOutputStream), but it's pretty straightforward -- you can just copy the code out of message.cc. Another thing that you could do which is specific to your use case: Instead of allocating a byte array to serialize into, just use a string: string bytes; pkt.SerializeToString(bytes); send(fd, bytes.data(), bytes.size(), 0); This is better anyway since it won't break if your messages become larger than SIZE. On Fri, Sep 19, 2008 at 2:12 PM, Leandro Lucarella [EMAIL PROTECTED] wrote: Hi. I guess SerializeToXxx() methods internally calls to ByteSize(), so after a serialization, I can call GetCachedSize(). My question is, first, is this true? =) And second, can I rely on this? The use case is this: pkt.SerializeToArray(buffer, SIZE); send(fd, buffer, pkt.GetCachedSize(), 0); Is this safe or should I use pkt.ByteSize() instead? Thank you. -- Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/ GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) Soñé que tenia un caballo Que me trataba mejor que vos Tenia tan buena onda con ella Era mi yegua, mucho mas que vos --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Issue dynamically reloading libprotobuf?
Unfortunately, the protocol buffer library was designed in an environment where all code is statically-linked, and it probably won't work so well with dynamically-loaded code. The problem is that at start-up, descriptors are built for each compiled-in message type and registered in a singleton registry. There is no code that removes the descriptors on shutdown. In your case, it looks like libprotobuf.so is staying loaded after your plugin is shut down. When it starts back up again, it tries to re-register its descriptors, but this fails because the descriptors already exist. You might be able to work around this by statically linking libprotobuf.a into your .so. That way, each time you load the library, a new registry will be created. You'll still have some memory leaks, though, as there are a lot of data structures which are not properly destroyed when libprotobuf is shut down (again, because the design assumed that the code would only be unloaded when the process exits anyway). It would probably be possible to rework parts of the library to make it more friendly to plugins, if someone wants to spend time on this, but it will take some effort. On Mon, Sep 22, 2008 at 8:53 PM, [EMAIL PROTECTED] wrote: Hi. I have an plug-in based application under Linux that dynamically links in libprotobuf.so. The app's plug-ins also link in the library. This works fine when a plug-in is first dynamically loaded via dl_open(). However, if the plug-in is removed dl_close() and then re- opened, I get the following error: libprotobuf ERROR google/protobuf/descriptor.cc:1778] Invalid proto descriptor for file component_ec.proto: libprotobuf ERROR google/protobuf/descriptor.cc:1781] component_ec.proto: A file with this name is already in the pool. It appears that there is some unwanted state associated with open/ close. Has anyone seen this type of issue? Any ideas? Thanks, Aaron --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: xlC suffers the same can't-decide-which-template-to-use as Symbian
It seems like something went wrong with the upload. When I look at the side-by-side diffs it just says error: old chunk mismatch. Do you have any idea what might have gone wrong? On Wed, Sep 24, 2008 at 6:31 PM, Travis Pouarz [EMAIL PROTECTED] wrote: On Sep 24, 7:17 pm, Kenton Varda [EMAIL PROTECTED] wrote: Thanks. Can you send this to me via codereview.appspot.com? http://codereview.appspot.com/6051 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Two test suite failures on AIX 5.3 / xlC 7.0
On Wed, Sep 24, 2008 at 4:37 PM, Travis Pouarz [EMAIL PROTECTED] wrote: This first one reveals a disagreement about whether a directory name should be followed by a slash: [ RUN ] CommandLineInterfaceTest.OutputDirectoryIsFileError google/protobuf/compiler/command_line_interface_unittest.cc:411: Failure Value of: printer.failed() Actual: true Expected: false google/protobuf/compiler/command_line_interface_unittest.cc:328: Failure Value of: expected_substring Actual: foo.proto/: Not a directory Expected: a substring of error_text_ Which is: output.test.foo.proto: Not a directory [ FAILED ] CommandLineInterfaceTest.OutputDirectoryIsFileError The test creates a file called foo.proto and then attempts to access foo.proto/, which should fail because the trailing slash implies that a directory is expected. I guess your platform just ignores the trailing slash. This isn't a big deal, since it just means protoc's errors might be a bit less helpful than they should be. I suppose you could #ifdef out the test on your platform with a comment saying it doesn't work correctly due to an OS bug but we don't care that much. Here's another failure, though the nature of the failure (and what the malfunction might affect) isn't so obvious: [ RUN ] DiskSourceTreeTest.MapDirectory google/protobuf/compiler/importer_unittest.cc:362: Failure Value of: input == NULL Actual: false Expected: true [ FAILED ] DiskSourceTreeTest.MapDirectory Not enough info. Can you change line 362 from: EXPECT_TRUE(input == NULL); to: EXPECT_TRUE(input == NULL) Expected file not found: filename; then run again? This should tell us exactly which file is being found when it shouldn't be. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Things to do with extensible options....
Interesting ideas. Before we get too creative, though, let me say a word of caution: if an option affects the generated code in all languages (e.g. by changing the wire format), then it can't really be a custom option, since it will have to be compiled into protoc anyway. I'm very wary about adding such features, because while they're often very useful in certain cases, together they make the system more bloated, possibly harming more than helping overall (since all users get the additional bloat but only some actually use the feature). With that said, here's some more ideas: * Cache keys: If you have a service which takes a protocol buffer request and returns a protocol buffer response with no side-effects, you could implement a caching layer over it. Cache keys could be generated by simply serializing the message and hashing it. However, your request messages might contain some fields which should be ignored when computing a cache key, e.g. options which only affect post-cache operations. You could use a custom option to mark which fields should be stripped before computing cache keys. * Auto interface generation: Someone mentioned before that they'd like to auto-generate a user interface for inputting or printing a protocol message in a friendly way. For this, it would be useful to have a custom string option called description which is a human-readable description of the field. On Thu, Sep 25, 2008 at 10:24 AM, Chris [EMAIL PROTECTED] wrote: Since there extensible options are almost released, I though I would start a thread to discuss what would be interesting to support. The current options affect the code generation trade offs for a given language: optional string java_package = 1; optional string java_outer_classname = 8; optional bool java_multiple_files = 10 [default=false]; optional OptimizeMode optimize_for = 9 [default=CODE_SIZE]; optional CType ctype = 1; Or are not to be used in current .proto files: optional bool message_set_wire_format = 1 [default=false]; optional string experimental_map_key = 9; My brainstorming at the moment produces the following language-agnostic suggestions. Note that I am not suggesting the names for any of these proposed options ( these bikesheds are unpainted ) : * FileOptions ** A version option that holds a string. This could be inserted into the generated code so that programs can inquire and report what version of the .proto file was used. ( Since protocol-buffers is designed to support changes in formats over time I am somewhat surprised this is not already present. ) ** Many of the MessageOptions or EnumOptions below could be set at the FileOptions level to affect all such messages or enums by default. * MessageOptions ** an option to control whether this message (by default) uses the length-prefix message format or the start/end tag group format on the wire. This would make people who prefer streaming writes over known-length writes happier. ** an option to control whether, when read from the wire format, this message should allow unknown fields or always throw an error. This might be handy to avoid much boilerplate checking for stray unknown fields. One could even make the message object slimmer if it does not not a pointer/reference to some unknown field data structure. ** an option to have the message skip unknown fields on the wire without either storing them or throwing an error. This might be handy to define cut-down versions of messages that skip over unwanted parts to save memory and speed up processing when loading (really querying) a large binary file. * EnumOptions ** An option to control the wire format, so it could be set to sfixed32 or sint32 by default. This would be an annoying change to support, so I probably should not have mentioned it. It makes large positive and all negative values more compact on the wire. ** An option to set the default value. The current default-default value is always the first enum value in the .proto file. * FieldOptions ** Note that the message options and enum option above may be considered to set the default behavior for the a message, but that a corresponding field option could be defined to override this for a given use of the message or enum. ( This is already possible for setting the default enum value for the field !) ** One could mark a repeated field to have a more efficient wire encoding: Write the field-id+wire-type tag only once followed by the uint32 encoded count of items (non-zero!) and then each item without any more of the tags. This would enable much more compact storage of numeric arrays. ** One could mark a repeated field to never have zero length. This might be handy to avoid much boilerplate checking for empty fields. ** One could mark an optional field to always write a value when serialized. So if it is not set it will cause the default value to be serialized into
MSVC build switched to static linking
Hi all, Just before building the 2.0.2 release candidate, I submitted a change to make static linking the default for the protobuf libraries under MSVC. This seems necessary to avoid some of the problems inherent in DLLs which have been discussed here, such as the fact that each DLL has a separate heap. You can still build DLLs by simply changing the libprotobuf and libprotoc projects to be DLL projects -- instructions are included in vsprojects/readme.txt. Let me know if anyone objects to this. I don't use MSVC much so it's hard for me to know what is best for MSVC users. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Can a text formated Flat file be parsed to PB message given the format of message in .proto file?
You can use TextFormat for this. Say your file is going to have a set of
FooMessages in it. Then, define a message like:
message FooSet {
repeated FooMessage foo = 1;
}
Now, your file can look like this:
foo {
... contents of the first FooMessage ...
}
foo {
... the second FooMessage ...
}
...
Now parse the entire file as a single FooSet.
BTW, protoc can do text-binary conversion from the command line:
protoc foo.proto --encode=FooSet text_data binary_data
Hope that helps.
On Sun, Oct 5, 2008 at 11:43 PM, Akhilesh [EMAIL PROTECTED] wrote:
More over, I could use TextFormat class for reading a single message
in TEXT format. But there is a restriction here, If I have multiple
messages in the same file, then it's not possible to create multiple
messages while parser is reading from input stream.
Please correct me if I am wrong.
Finally Aim is: If I have *multiple* messages written in *text* in any
file, I want to open a stream to read this file. And now I should be
able to simply wrap this stream in suitable PB streams and simple call
any function that will go on reading the stream and provide me an
array of PB Messages in Binary format as a result.
So is there anything like that? Or I'll have to write my code above PB
api?
Thanks in advance.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: Solaris 10 Compile problem
Hi Chad,
First, please be aware that protobuf@googlegroups.com is a public mailing
list with non-Googlers on it. If you need to communicate anything
confidential or Google-specific, please e-mail the appropriate internal
mailing list.
Anyway, I'm afraid you're going to have to help out with this, as I do not
have a Solaris machine, and bus error is way too generic to tell me
anything useful. Can you run the tests in a debugger and give me a stack
trace?
On Tue, Oct 7, 2008 at 2:51 PM, Chad Harvey [EMAIL PROTECTED] wrote:
Howdy -
I'm an SA (from doubleclick) working with some former doubleclick and other
google folx trying to get some things set up to xfer between legacy dclk and
google.
They requested that I install protobuf on a couple of hosts, but I'm having
some problem with the make check, and was hoping you could help.
If there's somewhere else I should ask about this (like a guts queue),
please let me know.
This is on a Solaris 10, sparc arch, T2000, with gcc/g++ 3.4.6.
The only errors I got on the compile were from ld at the end:
g++ -D_REENTRANT -pthreads -Wall -Wwrite-strings -Woverloaded-virtual
-Wno-sign-compare -g -O2 -D_REENTRANT -pthreads -o .libs/protobuf-test
common_unittest.o strutil_unittest.o descriptor_database_unittest.o
descriptor_unittest.o dynamic_message_unittest.o extension_set_unittest.o
generated_message_reflection_unittest.o message_unittest.o
reflection_ops_unittest.o repeated_field_unittest.o text_format_unittest.o
unknown_field_set_unittest.o wire_format_unittest.o coded_stream_unittest.o
printer_unittest.o tokenizer_unittest.o zero_copy_stream_unittest.o
command_line_interface_unittest.o importer_unittest.o parser_unittest.o
cpp_bootstrap_unittest.o cpp_unittest.o test_util.o googletest.o file.o
gtest.o gtest-death-test.o gtest-filepath.o gtest-port.o gtest_main.o
unittest.pb.o unittest_import.pb.o unittest_mset.pb.o
unittest_optimize_for.pb.o unittest_embed_optimize_for.pb.o
unittest_custom_options.pb.o cpp_test_bad_identifiers.pb.o
./.libs/libprotobuf.so
-L/sol10/SOURCES/S10/gcc-3.4.6/objdir/sparc-sun-solaris2.10/libstdc++-v3/src
-L/sol10/SOURCES/S10/gcc-3.4.6/objdir/sparc-sun-solaris2.10/libstdc++-v3/src/.libs
-L/usr/local/lib -L/usr/local/ssl/lib -L/usr/openwin/lib
-L/sol10/SOURCES/S10/gcc-3.4.6/objdir/gcc ./.libs/libprotoc.so
/root/protobuf-2.0.2/src/.libs/libprotobuf.so -lpthread
/usr/local/lib/libstdc++.so -Wl,-R -Wl,/usr/local/lib
ld: warning: file /root/protobuf-2.0.2/src/.libs/libprotobuf.so: linked to
./.libs/libprotobuf.so: attempted multiple inclusion of file
ld: warning: file /usr/local/lib/libstdc++.so: attempted multiple inclusion
of file
creating protobuf-test
make[3]: Leaving directory `/root/protobuf-2.0.2/src'
make[2]: Leaving directory `/root/protobuf-2.0.2/src'
make[2]: Entering directory `/root/protobuf-2.0.2'
make[2]: Leaving directory `/root/protobuf-2.0.2'
make[1]: Leaving directory `/root/protobuf-2.0.2'
The 'make check' fails with:
Making check in src
make check-am
make check-TESTS
Running main() from gtest_main.cc
[==] Running 557 tests from 85 test cases.
[--] Global test environment set-up.
[--] 1 test from GeneratedDescriptorTest
[ RUN ] GeneratedDescriptorTest.IdenticalDescriptors
[ OK ] GeneratedDescriptorTest.IdenticalDescriptors
[--] 22 tests from GeneratedMessageTest
[ RUN ] GeneratedMessageTest.Defaults
[ OK ] GeneratedMessageTest.Defaults
[ RUN ] GeneratedMessageTest.Accessors
[ OK ] GeneratedMessageTest.Accessors
[ RUN ] GeneratedMessageTest.MutableStringDefault
[ OK ] GeneratedMessageTest.MutableStringDefault
[ RUN ] GeneratedMessageTest.Clear
[ OK ] GeneratedMessageTest.Clear
[ RUN ] GeneratedMessageTest.EmbeddedNullsInBytesCharStar
[ OK ] GeneratedMessageTest.EmbeddedNullsInBytesCharStar
[ RUN ] GeneratedMessageTest.ClearOneField
[ OK ] GeneratedMessageTest.ClearOneField
[ RUN ] GeneratedMessageTest.CopyFrom
[ OK ] GeneratedMessageTest.CopyFrom
[ RUN ] GeneratedMessageTest.CopyConstructor
[ OK ] GeneratedMessageTest.CopyConstructor
[ RUN ] GeneratedMessageTest.CopyAssignmentOperator
[ OK ] GeneratedMessageTest.CopyAssignmentOperator
[ RUN ] GeneratedMessageTest.UpcastCopyFrom
[ OK ] GeneratedMessageTest.UpcastCopyFrom
[ RUN ] GeneratedMessageTest.DynamicMessageCopyFrom
bash: line 4: 9616 Bus Error (core dumped) ${dir}$tst
FAIL: protobuf-test
==
1 of 1 tests failed
Please report to protobuf@googlegroups.com
==
*** Error code 1
The following command caused the error:
failed=0; all=0; xfail=0; xpass=0; skip=0; \
srcdir=.; export srcdir; \
list='protobuf-test'; \
if test -n $list; then \
for tst in $list; do \
if test -f ./$tst; then dir=./; \
elif test -f $tst; then dir=;
Re: Import broken?
protobuf_BuildDesc_RequestHeader_2eproto() should be declared in
RequestHeader.pb.h. Is it not?
What exact flags are you passing to protoc?
On Thu, Oct 9, 2008 at 8:29 AM, [EMAIL PROTECTED] wrote:
I'm in the process of evaluating PB for a distributed cross language
project I'm on. I've created a little 'hello world' app to test it's
capabilities. I have created 2 simple .proto files,
RequestHeader.proto and Rerender.proto. They compile and run as
expected. However, if I import RequestHeader into Rerender with the
following line:
import RequestHeader.proto;
I get the following compile error:
Error C2039: 'protobuf_BuildDesc_RequestHeader_2eproto' : is not a
member of '`global namespace''
The method protobuf_BuildDesc_RequestHeader_2eproto() doesn't exist
anywhere in the code except where it errors out. Below is the
offending method in Rerender.pb.cc that contains the error:
void
protobuf_BuildDesc_e_3a_2fVital_203_2e0_2fCPPProtobufTest_2fCPPProtobufTest_2fRerender_2eproto()
{
static bool already_here = false;
if (already_here) return;
already_here = true;
GOOGLE_PROTOBUF_VERIFY_VERSION;
::google::protobuf::DescriptorPool* pool =
::google::protobuf::DescriptorPool::internal_generated_pool();
::protobuf_BuildDesc_RequestHeader_2eproto(); // -- ERROR
pool-InternalBuildGeneratedFile(
\n;e:/Vital 3.0/CPPProtobufTest/CPPProtob
ufTest/Rerender.proto\032\023RequestHeader.pro
to\\030\n\010Rerender\022\014\n\004Data\030\001 \002(\t, 108,
protobuf_BuildDesc_e_3a_2fVital_203_2e0_2fCPPProtobufTest_2fCPPProtobufTest_2fRerender_2eproto_AssignGlobalDescriptors);
}
I would assume that the ability to import one proto file into another
is something that is exercised quite often, so I'm at a loss as to why
the resulting code from protoc won't compile. I'm using protobuf 2.0.2
built from source using VS 2008.
Any assistance on how to get this working would be greatly
appreciated. Thanks.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: Import broken?
On Thu, Oct 9, 2008 at 12:52 PM, [EMAIL PROTECTED] wrote: protoc.exe --cpp_out= [Fully qualified path name] Ah, ok, you've identified a bug. protoc is supposed to reject any input file which is not located within the import path. The import path is specified using --proto_path and defaults to the current directory. The problem is that protoc detects whether an input file's path is absolute based on whether it starts with a slash, but on Windows absolute paths don't start with a slash. So, protoc failed to report the incorrect input path. To fix this, you can either do: protoc --proto_path=[full path of current directory] --cpp_out=. [full path of input file] Or do: protoc --cpp_out=. [file name without path] I'll make sure the bug gets fixed. Thanks. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: How can I compile the protobuf with stlport?
I'm not familiar with stlport. One thing you could try doing is editing config.h manually and removing all the HASH_* #defines. Then the code will fall back to using map instead of hash_map, which should work since map has a standard interface. On Fri, Oct 10, 2008 at 2:58 AM, [EMAIL PROTECTED] wrote: I'm using stlport, and i want to use protobuf, but I see some compiling errors about the hash_map. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Merge two messages into one file
Sorry, I'm not sure I understand the question. What, exactly, is the main
problem you're having?
On Fri, Oct 10, 2008 at 3:11 AM, [EMAIL PROTECTED]
[EMAIL PROTECTED]wrote:
I have some kind of requirement that I did not find answer from the
document. May someone help me.
I have different message objects which are inherited from one parent.
I need to serialize the objects into one file.
Some kind of solutions like:
e.g.
message Style {
optional int32 color = 3;
optional int32 size = 4;
}
message Text {
optional Style style = 3;
optional string context = 4;
}
message Image {
optional Style style = 3;
optional int32 width = 4;
optional int32 height = 5;
optional bytes data = 6;
}
My Group object includes Text objects and Image objects. But the
position is not fixed.
e.g.Text.Image.Image...Text
and could be: Image.Text...Image...Text
Now I need to serialize the objects into one file, and seek for the
solution using protocol buffer.
Implement this in Java is quite easy, because I can use
DataInputStream and serialize the objects.
Thanks for any hints.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: invalid tag (zero)
Are you sure that the data you are sending to the parser is exactly the same data that was generated by the serializer? Remember that protocol buffers are not self-delimiting, so you need to make sure that you limit the input to the exact number of bytes that were produced when serializing. If the data is exactly the same, then this is a bug. If you can create a small program or pair of programs that demonstrate the problem, I would be happy to debug it. On Mon, Oct 13, 2008 at 10:09 AM, Dominik Steenken [EMAIL PROTECTED] wrote: Hi everyone, we are currrently implementing a server/client system, the server being implemented in c++, the client in java. During our last rounds of tests, we encountered a problem that had to do with the sending of (not so) long messages. on the (receiving) java side, we get the following exception: Exception in augnet.client.aim.connection.Receiver, Parse error: com.google.protobuf.InvalidProtocolBufferException: Protocol message contained an invalid tag (zero). com.google.protobuf.InvalidProtocolBufferException: Protocol message contained an invalid tag (zero). at com.google.protobuf.InvalidProtocolBufferException.invalidTag(InvalidProtocolBufferException.java: 52) at com.google.protobuf.CodedInputStream.readTag(CodedInputStream.java: 67) at com.google.protobuf.FieldSet.mergeFrom(FieldSet.java:397) at com.google.protobuf.AbstractMessage $Builder.mergeFrom(AbstractMessage.java:248) at com.google.protobuf.GeneratedMessage $Builder.mergeFrom(GeneratedMessage.java:1) at com.google.protobuf.CodedInputStream.readMessage(CodedInputStream.java: 227) at com.google.protobuf.FieldSet.mergeFieldFrom(FieldSet.java:482) at com.google.protobuf.FieldSet.mergeFrom(FieldSet.java:402) at com.google.protobuf.AbstractMessage $Builder.mergeFrom(AbstractMessage.java:248) at com.google.protobuf.GeneratedMessage $Builder.mergeFrom(GeneratedMessage.java:1) at com.google.protobuf.CodedInputStream.readMessage(CodedInputStream.java: 227) at com.google.protobuf.FieldSet.mergeFieldFrom(FieldSet.java:482) at com.google.protobuf.FieldSet.mergeFrom(FieldSet.java:402) at com.google.protobuf.AbstractMessage $Builder.mergeFrom(AbstractMessage.java:248) at com.google.protobuf.AbstractMessage $Builder.mergeFrom(AbstractMessage.java:240) at com.google.protobuf.AbstractMessage $Builder.mergeFrom(AbstractMessage.java:298) at augnet.client.aim.messages.MessageProtos $AugNetMessage.parseFrom(MessageProtos.java:6289) at augnet.client.aim.connection.Receiver.run(Receiver.java:47) while the (sending) c++ side encounters no errors. When we scale down the message, no error occurs. Is this a bug in protobuf or are we doing something wrong? Best regards, Dominik --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: 2.0.2 release is up
On Mon, Oct 13, 2008 at 2:26 AM, edan [EMAIL PROTECTED] wrote: I quick google didn't turn up much for me on what that actually means - does it mean anything to y'all? Yes, this was an overzealous warning present in one version of GCC which was promptly toned down in subsequent versions. You can ignore it. In any case, using gcc 4.1.2, make and make check (any reason you didn't use the more standard make test?) succeeded for me, so I guess I will have to just wait to update to protobuf-2.0.2 until I can move myself to the newer gcc. I'll try to look into this more when I get a chance. You should be fine sticking with 2.0.1, though, as long as you weren't hoping to use the new custom options feature. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: How can I compile the protobuf with stlport?
Does stlport's hash_map work like SGI's hash_map, or like MSVC's hash_map, or does it do its own thing? A brief look at the docs suggest that it is most similar to SGI's. In that case, you should only need to change the #ifdefs in src/google/protobuf/stubs/hash.h so that your compile takes the non-MSVC branch. That is, change line 101 from: #elif defined(_MSC_VER) to: #elif 0 Let me know if this works. On Mon, Oct 13, 2008 at 3:08 AM, [EMAIL PROTECTED] wrote: thanks. it's okay, but i am thinking how to use stlport's hash_map with protobuf. how can i do? On 10月11日, 上午4时24分, Kenton Varda [EMAIL PROTECTED] wrote: I'm not familiar with stlport. One thing you could try doing is editing config.h manually and removing all the HASH_* #defines. Then the code will fall back to using map instead of hash_map, which should work since map has a standard interface. On Fri, Oct 10, 2008 at 2:58 AM, [EMAIL PROTECTED] wrote: I'm using stlport, and i want to use protobuf, but I see some compiling errors about the hash_map. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Unknown Message Type
You can create a message like this:
message Container {
// One of the following will be filled in.
optional Ping ping = 1;
optional StatusUpdate status_update = 2;
optional FindWorker find_worker = 3;
}
Now send Containers over the wire instead of instances of the individual
message types.
On Tue, Oct 14, 2008 at 11:37 AM, [EMAIL PROTECTED] wrote:
Hi,
I was wondering if it was possible to extract the contents of a
serialized string without knowing ahead of time what kind of message
it is?
For example: I'm using tcp to send data back and fourth. On one side
I create a message and serialize the data, and on the receiving side I
'ParseFromString' and check the information. However, I have
different types of messages being sent so the receiving side has no
idea what it is actually getting.
example .proto file [messages actually contain field, taken out for
simplicity] which creates 'dtm_pb2' after compiling
package dtm;
message ping {
}
message statusUpdate {
}
message findWorker {
}
so in my code on the receiving end I have:
msg = dtm_pb2()
msg.ParseFromString(message)
this gives an error.
Is there any way to initialize the msg variable and extract the
information??
If I knew what kind of message it was I could always use:
msg = dtm_pb2.statusUpdate()
msg.ParseFromString(message)
and this would work fine, but this is no a viable solution for me.
thanks,
Nima
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: Popping values from repeated fields
Currently, no, you can only clear the whole list and start over. I would be amenable to a patch, though, if people feel this would be a useful addition. On Wed, Oct 15, 2008 at 8:36 PM, Andrew Wachira [EMAIL PROTECTED] wrote: In the Java language. On 10/15/08, Kenton Varda [EMAIL PROTECTED] wrote: In which language? On Wed, Oct 15, 2008 at 1:50 AM, Andy [EMAIL PROTECTED] wrote: Hi, We can push values onto repeated fields. Is there a way of popping values from the same? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: How should I Input the path of compiling .proto files?
You need to specify a --proto_path, like: protoc --proto_path=C:\Projects\Apps_main\C++\Common --cpp_out=. C:\Projects\Apps_main\C++\Common\test.proto There is a bug in protoc which causes it to print the wrong error here -- it should have reported that your input file was not in the proto_path. That will be fixed in the next version. On Thu, Oct 16, 2008 at 9:41 AM, [EMAIL PROTECTED] wrote: I have the same problem. Tried 2.0.2 release, the problem still exist. Here the error: protoc.exe --cpp_out=. C:\Projects\Apps_main\C++\Common\test.proto C:/Projects/Apps_main/C++/Common/test.pb.h: while trying to create directory ./C:: Invalid argument C:/Projects/Apps_main/C++/Common/test.pb.cc: while trying to create directory ./C:: Invalid argument On Sep 2, 2:22 pm, Kenton Varda [EMAIL PROTECTED] wrote: On Tue, Sep 2, 2008 at 10:06 AM, [EMAIL PROTECTED] wrote: the console display: Unknow generator option: d This is a bug which will be fixed in 2.0.1 which should be released any day now. You can get the release candidate here: http://groups.google.com/group/protobuf/files --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Protocol Buffer as XCode Framework
I'm not sure how mac frameworks work, but it looks like your compiler is not linking against the protobuf library. On Thu, Oct 16, 2008 at 9:41 AM, Carlos Gamboa [EMAIL PROTECTED] wrote: Hello. I am trying to add the protocol buffer to my application (for iphone) and i have been struggling for about 2 days to figure out how to do it. I am new to XCode too by the way. I tried creating a framework like Brian Olson suggested on this post: http://groups.google.com/group/protobuf/browse_thread/thread/65e781a5e8d6fc06# , but i do not know all the things i need to include this framework. Here are the steps im following. 1. Compile the protobuf project 2. On my project, right click on Frameworks and click 'add-existing frameworks' and then locate the 'macosx/build/release/ protobuf.framework' folder. The framework is located at the documents folder (documents/macosx) 3. I created my .pb.h and .pb.c with the 'protoc' program and added them to my project. 4. I try to compile, but the following 18 errors appear like this: google::protobuf::UnknownFieldSet::UnknownFieldSet(), referenced from: myNamespace::myProto::myProto() in myProto.pb.o ... I really dont care if i add it as a framework or as direct code, but i have not been able to include the protocol buffer as part of my project. Can anyone please help me? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: using RPC on asynchronous server
Many (most?) servers at Google use asynchronous protobuf RPC. I'm not sure how to answer your question, though. Can you be more specific? On Tue, Oct 14, 2008 at 7:16 AM, muqu [EMAIL PROTECTED] wrote: Anybody can shine some light on their experience on using the PB RPC on an asynchronous server, like whether RPC is well-designed for it or any potential pitfalls? I find the for async server, matching a response to the original request is kind of a pain. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Patch to quiet compiler warnings from generated service code
I've made this change in SVN. Thanks.
On Thu, Oct 2, 2008 at 12:40 PM, Caleb [EMAIL PROTECTED] wrote:
When run with high warning levels, gcc complains that the 'request'
and 'response' arguments are unused for all of the stub service
functions generated by the protoc compiler. Here's the 2-line patch
to fix this.
--- src/google/protobuf/compiler/cpp/cpp_service.cc.orig
2008-10-02 15:36:56.0 -0400
+++ src/google/protobuf/compiler/cpp/cpp_service.cc 2008-10-02
15:37:07.0 -0400
@@ -219,8 +219,8 @@
printer-Print(sub_vars,
void $classname$::$name$(::google::protobuf::RpcController*
controller,\n
- const $input_type$* request,\n
- $output_type$* response,\n
+ const $input_type$*,\n
+ $output_type$*,\n
::google::protobuf::Closure* done)
{\n
controller-SetFailed(\Method $name$() not implemented.\);
\n
done-Run();\n
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: protocol buffers not repeating items marked repeated
I don't see any obvious problems. Try printing the whole message just
before serializing and just after parsing to see if they are both as you
expect.
On Sat, Oct 18, 2008 at 11:24 PM, comsatcat [EMAIL PROTECTED] wrote:
I have a simple client server in python... I have the following
protobuf definition:
message NodeListData {
enum State {
FALSE = 0;
TRUE = 1;
}
required string node = 1;
required string folder = 2;
required State pendingcsr = 3;
}
message NodeList {
repeated NodeListData items = 1;
}
message Packet {
enum Type {
ACK = 0;
KEEP_ALIVE = 1;
AUTHORIZE = 2;
ADD_ENDPOINT = 3;
GET_ENDPOINT_STATS = 4;
ENDPOINT_STATS = 5;
REMOVE_ENDPOINT = 6;
NODE_LIST = 7;
}
required Type type = 1;
optional Ack ack = 2;
optional KeepAlive keepalive = 3;
optional Authorize authorize = 4;
optional AddEndpoint addendpoint = 5;
optional EndpointStats endpointstats = 6;
optional RemoveEndpoint removeendpoint = 7;
optional NodeList nodelist = 8;
}
*note some fields have been removed for shortening this message*
My problem is with the NodeList type...
I've verified in my server that I am doing a
packet.nodelist.items.add(), then proceeding to set the required
values of the returned NodeItemList.
I then send the packet over the wire.
I receive the packet from the client and do a:
for i in packet.nodelist.items:
print str(i.node)
and it only comes back with 1 result, no matter how many I build in
the server (I've tried up to 3)...
Any ideas?
I'm doing this exactly the same way with a different packet type and
it works fine... am I just missing something small?
Thanks,
Ben
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: cross compile errors
For the first case, the problem is that the build is trying to run protoc as part of the build process, and it's trying to use the cross-compiled protoc. I think you can fix this by editing src/Makefile.am at line 201 to change this: unittest_proto_middleman: protoc$(EXEEXT) $(protoc_inputs) ./$^ -I$(srcdir) --cpp_out=. to this: unittest_proto_middleman: $(protoc_inputs) protoc $^ -I$(srcdir) --cpp_out=. In other words, this will use the protoc that you have already installed on your system rather than the one you just built. I guess we should figure out how to make this automatic. On Tue, Oct 21, 2008 at 8:42 AM, [EMAIL PROTECTED] wrote: I'm trying to cross compile for a ppc platform from x86 Linux. I've used the same procedure as I used to successfully cross compile other tools (not to say that the procedure is actually right) but I get some errors. Here's what I tried: setenv CC /opt/cross_compiler/ppc/gcc-4.1.0-glibc-2.3.6/powerpc-603- linux-gnu/bin/powerpc-603-linux-gnu-gcc setenv CXX /opt/cross_compiler/ppc/gcc-4.1.0-glibc-2.3.6/powerpc-603- linux-gnu/bin/powerpc-603-linux-gnu-g++ setenv PATH $PATH:/opt/cross_compiler/ppc/gcc-4.1.0-glibc-2.3.6/ powerpc-603-linux-gnu/bin cd /tmp/pbmpc5200/protobuf-2.0.2rc1/ ./configure --target=ppc-603-linux-gnu --host=ppc-603-linux-gnu -- build=i686 make Here are the errors: make[2]: Entering directory `/tmp/pbmpc5200/protobuf-2.0.2rc1/src' ./protoc google/protobuf/unittest.proto google/protobuf/ unittest_import.proto google/protobuf/unittest_mset.proto google/ protobuf/unittest_optimize_for.proto google/protobuf/ unittest_embed_optimize_for.proto google/protobuf/ unittest_custom_options.proto google/protobuf/compiler/cpp/ cpp_test_bad_identifiers.proto -I. --cpp_out=. ./protoc: line 105: /tmp/pbmpc5200/protobuf-2.0.2rc1/src/.libs/lt- protoc: cannot execute binary file ./protoc: line 105: /tmp/pbmpc5200/protobuf-2.0.2rc1/src/.libs/lt- protoc: Success make[2]: *** [unittest_proto_middleman] Error 1 make[2]: Leaving directory `/tmp/pbmpc5200/protobuf-2.0.2rc1/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/tmp/pbmpc5200/protobuf-2.0.2rc1' make: *** [all] Error 2 The libprotobuf.so that I wanted actually was actually created and works, but there were errors reported. Did I configure incorrectly or are these errors just expected when cross compiling? I looks like it might be trying to run a cross compiled binary? Similarly, when cross compiling to win32 using mingw, I get an error: make[2]: Entering directory `/tmp/pbmingw/protobuf-2.0.2rc1/src' /bin/sh ../libtool --tag=CXX --mode=link /opt/cross-tools/bin/i386- mingw32msvc-g++ -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign- compare -g -O2 -o protoc.exe main.o libprotobuf.la libprotoc.la /opt/cross-tools/bin/i386-mingw32msvc-g++ -Wall -Wwrite-strings - Woverloaded-virtual -Wno-sign-compare -g -O2 -o protoc.exe main.o ./.libs/libprotobuf.a -L/tmp/mingwtmp/gcc-i386-mingw32msvc/ i386-mingw32msvc/libstdc++-v3/src -L/tmp/mingwtmp/gcc-i386-mingw32msvc/ i386-mingw32msvc/libstdc++-v3/src/.libs -L/tmp/mingwtmp/gcc-i386- mingw32msvc/gcc -L/opt/cross-tools/i386-mingw32msvc/bin -L/opt/cross- tools/i386-mingw32msvc/lib -L/opt/cross-tools/lib/../i386-mingw32msvc/ lib ./.libs/libprotoc.a /tmp/pbmingw/protobuf-2.0.2rc1/src/.libs/ libprotobuf.a /opt/cross-tools/i386-mingw32msvc/lib/libstdc++.a -lm ./.libs/libprotobuf.a: could not read symbols: Archive has no index; run ranlib to add one collect2: ld returned 1 exit status make[2]: *** [protoc.exe] Error 1 make[2]: Leaving directory `/tmp/pbmingw/protobuf-2.0.2rc1/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/tmp/pbmingw/protobuf-2.0.2rc1' make: *** [all] Error 2 Manually running ranlib on the library seems to work. So in both cases, the builds gave errors but I was able to get the libraries I needed. Thanks, David --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: importing across dlls
I believe this was fixed in 2.0.2. What version are you using? On Tue, Oct 21, 2008 at 11:21 AM, [EMAIL PROTECTED] wrote: I have a protoc file named UniqueID.proto that is compiled into and exported from Communication.dll. I have a protoc file named Base.proto that is compiled into and exported from Base.dll. Base.proto imports and uses UniqueID.proto. Base.dll imports the UniqueID class. Importing and exporting is done via a macro included via the -- cpp_out=dllexport_decl= command line option. Heres the problem: Base.pb.cc tries to call protobuf_BuildDesc_UniqueID_2eproto(), which doesn't use the macro passed on the command line. The result is a linker error. Is it possible to import one .proto file into another when the classes they create will be compiled into different dlls? Thanks in advance!! --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: importing across dlls
On Tue, Oct 21, 2008 at 12:13 PM, [EMAIL PROTECTED] wrote: Would simply changing FileGenerator::GenerateHeader to output the dllexport_decl in the forward declaration of BuildDescriptors function work? Yes, that is the fix. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Compressed protobufs?
There is no built-in support for this, but there's nothing stopping you from applying whatever compression you want to the serialized data. On Tue, Oct 21, 2008 at 1:08 PM, GDR [EMAIL PROTECTED] wrote: I noticed the protocol buffers store strings in their uncompressed form. Is there any provision to produce and consume protobufs in compressed form? This would be specially helpful if a lot of fields were repeated strings. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Make check reports error
On Wed, Oct 22, 2008 at 7:23 AM, Niclas Blomgren [EMAIL PROTECTED] wrote: Solaris: I'm guessing this was a sparc system, not x86? Was it 32-bit or 64-bit? I think someone else reported the same problem but we were not able to track it down. Cygwin: Can you include the text of the actual failures in your log? (You only included the summary.) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Serializing Large Collections: SerializePartial ParsePartial
The Partial serialize and parse routines actually do something completely unrelated: they allow the message to be missing required fields. So, that doesn't help you. I'm afraid protocol buffers are not designed for storing very large collections in a single message. Instead, you should be thinking about representing each item in a collection using a protocol message, but then using some other container format. Using protocol buffers here makes the container format simpler because it only needs to deal with raw strings rather than having to worry about structured data. On Wed, Oct 22, 2008 at 10:19 AM, [EMAIL PROTECTED] wrote: Is there a general strategy/methodology for dealing with very large collections so that they do not need to be completely held in memory before serializing and de-serializing? I see, for example, SerializePartialToOstream() and ParsePartialFromIstream() but no documentation of how to use it. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Make check reports error
Cygwin allows you to specify paths like C:\. For example, try this
command:
ls 'C:\'
At least, for me, this lists the root of the C drive. It looks like the
test failed because you do not have permission to access C:\ on your
machine. Is this possible? You can probably ignore this failure.
The sparc failure is still mysterious to me. Unfortunately I don't have a
sparc machine on which I can debug it. :(
On Thu, Oct 23, 2008 at 1:22 AM, Niclas Blomgren
[EMAIL PROTECTED] wrote:
Hi again,
I took a quick look at this code in command_line_interface_unittest.cc
TEST_F(CommandLineInterfaceTest, WindowsOutputPath) {
// Test that the output path can be a Windows-style path.
NullCodeGenerator* generator = RegisterNullGenerator(--test_out);
CreateTempFile(foo.proto,
syntax = \proto2\;\n);
Run(protocol_compiler --test_out=C:\\
--proto_path=$tmpdir foo.proto);
ExpectNoErrors();
EXPECT_TRUE(generator-called_);
EXPECT_EQ(, generator-parameter_);
}
At a quick glance it seems like the code tries to access C:.
In cygwin however the path to C: is /cygdrive/c.
BR / N
--
*From:* Niclas Blomgren
*Sent:* den 23 oktober 2008 09:50
*To:* 'Kenton Varda'
*Cc:* protobuf@googlegroups.com
*Subject:* RE: Make check reports error
Hi!
Solaris: Yes it was a sparc system. 64-bit.
Cygwin: Ok here it is. I have also attached the entire make check in a
logfile.
[ RUN ] CommandLineInterfaceTest.WindowsOutputPath
google/protobuf/compiler/command_line_interface_unittest.cc:329: Failure
Value of: return_code_
Actual: 1
Expected: 0
google/protobuf/compiler/command_line_interface_unittest.cc:330: Failure
Value of: error_text_
Actual: C:\/: Permission denied
Expected:
google/protobuf/compiler/command_line_interface_unittest.cc:522: Failure
Value of: generator-called_
Actual: false
Expected: true
[ FAILED ] CommandLineInterfaceTest.WindowsOutputPath
[ RUN ] CommandLineInterfaceTest.WindowsOutputPathAndParameter
google/protobuf/compiler/command_line_interface_unittest.cc:329: Failure
Value of: return_code_
Actual: 1
Expected: 0
google/protobuf/compiler/command_line_interface_unittest.cc:330: Failure
Value of: error_text_
Actual: C:\/: Permission denied
Expected:
google/protobuf/compiler/command_line_interface_unittest.cc:538: Failure
Value of: generator-called_
Actual: false
Expected: true
google/protobuf/compiler/command_line_interface_unittest.cc:539: Failure
Value of: generator-parameter_
Actual:
Expected: bar
[ FAILED ] CommandLineInterfaceTest.WindowsOutputPathAndParameter
Hopeing you can help me.
Best Regards / Niclas Blomgren
--
*From:* Kenton Varda [mailto:[EMAIL PROTECTED]
*Sent:* den 23 oktober 2008 00:19
*To:* Niclas Blomgren
*Cc:* protobuf@googlegroups.com
*Subject:* Re: Make check reports error
On Wed, Oct 22, 2008 at 7:23 AM, Niclas Blomgren
[EMAIL PROTECTED] wrote:
Solaris:
I'm guessing this was a sparc system, not x86? Was it 32-bit or 64-bit? I
think someone else reported the same problem but we were not able to track
it down.
Cygwin:
Can you include the text of the actual failures in your log? (You only
included the summary.)
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: Where are the pre-compiled lib and supporting header files?
We do not provide a pre-compiled library because it is important that you use a version of the library compiled with the same compiler version which you will be using with your own code. Even on Windows, different versions of MSVC have slightly different STL implementations. Since the protocol buffer library uses STL in its interface, the app and the library have to be using identical STL implementations in order to be compatible. So, please just use your own compiler to compile the protobuf library. See the readme files for step-by-step instructions. On Thu, Oct 23, 2008 at 2:06 AM, sdx [EMAIL PROTECTED] wrote: Hi, I would be grateful if someone could point me in the right direction (please see subject) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Any way to dissect ProtBuf serialized data without knowing the structure in advance?
From the command line, you can do: protoc --decode_raw data_file This will give you as much information as can be determined without a type definition, to give you an idea of what's there. In C++, you can use the UnknownFieldSet class to parse and inspect a message of unknown type. On Wed, Oct 22, 2008 at 10:32 PM, [EMAIL PROTECTED] wrote: I'm trying to consume data from an app that generates output serialized via Protocol Buffers but do not have the original spec for the specific structures that have been encoded. Is there a relatively straight-forward path to deserializing, or even just decoding, the serialized data stream without knowing its structure in advance? Hints/pointers of any variety would be welcomed. Thanks. APB --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: strings with fixed length
STL strings never shrink. When they're cleared, they keep their memory
around for reuse. So, you could take advantage of that to accomplish what
you want by setting all your string fields to values of the maximum size and
then clearing them. For example:
StaticSizeMsg msg;
msg.set_message(string(50, 'x'));
msg.Clear();
Now if you use msg to parse any message where the message field is = 50
bytes, it will not do any allocation.
On Thu, Oct 23, 2008 at 4:17 AM, thecreator [EMAIL PROTECTED] wrote:
Hi!
I'd like to realize messages which do not need to allocate memory
during runtime. All the allocation should be done
at initialization. Therefor I'd need some mechanism to define strings
with fixed length in .proto files.
I read about the option annotation in version 2.0.2.
Is it possible to change the code generation with this annotations, so
that a string is not created as new std::string
but also reserve the memory which is set in the option:
e.g. in the proto file:
message StaticSizeMsg
{
required string message = 1 [size = 50];
}
The generated code should like like this:
std::string message_;
message_.reserver(50);
thanks,
greets Ralph
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: Streaming
In general, protocol buffers can be used to encode individual chunks within a stream, but representing the entire stream as a protocol buffer won't work very well. On Thu, Oct 23, 2008 at 8:14 PM, sureshkk [EMAIL PROTECTED] wrote: Hi, I am looking at a use case where one needs to transfer streaming data (video, file transfer etc..). Will PB support such a use case on its own? Thanks Best Regards, Suresh --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Zero enums
The reasoning for unknown enums being treated as unknown fields goes something like this: We cannot simply use an unknown numeric value since many languages do not allow enum types to represent numeric values other than the set of values explicitly defined for them. Furthermore, even if they did, this could lead to security problems: people might write server code which assumes that an enum field has one of the defined values and does something bad if it isn't. Such problems could easily get past code review and testing. On the other hand, in the case of servers which simply act as a proxy, forwarding messages elsewhere, we do not want to have to upgrade these servers every time a new enum value is added to the proto definition. Instead, we would like these servers to be able to pass the message through retaining the value they read in, valid or not. This is exactly why unknown fields exist in the first place, so treating it as an unknown field seems appropriate. On Fri, Oct 24, 2008 at 1:53 PM, Marc Gravell [EMAIL PROTECTED]wrote: I might have found the answer... if you try to provide a different value, the parser will treat it like an unknown field (from the language guide). So this means that enums are essentially unchecked: invalid values are silently ignored? Marc --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Zero enums
Oh, the implicit default for enums is the first defined value, not zero. On Fri, Oct 24, 2008 at 3:03 PM, Marc Gravell [EMAIL PROTECTED]wrote: Makes sense - it just seems a little odd that the optional enums don't have a valid default... Marc --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Zero enums
Added to my todo list. On Sat, Oct 25, 2008 at 1:19 AM, Marc Gravell [EMAIL PROTECTED]wrote: Ah! Right. That makes more sense. Could that perhaps be added to the language guide Optional Fields And Default Values? Marc --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Problem with spaces in path on windows
Can you give a specific example of a complete protoc command and the error it produces? protoc should not care about spaces in file names as long as they are properly quoted. On Mon, Oct 27, 2008 at 6:49 AM, sdx [EMAIL PROTECTED] wrote: Hi, While trying to invoke the protoc compiler on the windows platform on a file placed in a path containing spaces the compiler returns with an error saying that the given path was not found (it shows an incomplete path up until the first space in the path as part of the error message) I am using the following batch file as a work around for now but I hope the maintainers will rectify this error in any case. @echo off ::File: Runprotoc.cmd ::%1 = Path to source .proto file ::push the current dir onto the directory stack and change to the input file's directory pushd %~dp1 echo protoc --cpp_out=. %~nx1 protoc --cpp_out=. %~nx1 ::revert to the directory in use before entering the script popd Regards, Sandeep Datta --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: JSON format for ProtocolBuffers
It would be pretty easy to write code that serializes or parses java
protocol message objects in JSON format using the protobuf reflection
interface (see the Message and Builder interfaces). I don't know of any
existing implementations, though.
On Sun, Oct 26, 2008 at 2:11 PM, Richard [EMAIL PROTECTED] wrote:
Hi,
Does anyone have JsonFormat class for ProtocolBuffers in Java? Why
would this be useful? It would mean that that you could make rpc calls
from javascript to your web service that you implemented using
protocol buffers and jetty.
As an example lets say you have a message with type:
message Person {
required string name = 1;
repeated string phoneNumber = 2;
}
Examples of this message in Json are:
{ name: Richard }
{ name: Richard, phone: [ 555-555-111 ] }
See that repeated fields become lists. That repeated fields don't have
to be present if they're empty. The Json format doesn't have field
numbers. Our purpose is not to make the Json messages compact, it is
to make it easy to use them from javascript.
Embedded message work as you would expect:
message Person {
required string name = 1;
repeated string phoneNumber = 2;
repeated Address address = 3;
}
message Address {
required string addrLine1 = 1;
optional string addrLine2 = 2;
required string state = 3;
}
Then the Json is
{ name: Richard, phoneNumber: [ 555-555-555 ], address:
{ addrLine1 = 12 Somestreet, state = ZZ } }
regards,
Richard.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: reading one message at a time
The protocol buffer format expects you to remember where the message ends;
it cannot figure out for itself. So, you need to write the size of each
message to your file before you write the message itself.
On Mon, Oct 27, 2008 at 11:42 AM, Amit Gupta [EMAIL PROTECTED] wrote:
I have message defined as
message Person
{
required int32 id;
}
and than, after protoc-compiling I dump 500 Million of such buffers
using a c++ application into a file.
Now, when I read it back using a different C++ application (and
ParseFromIoStream, I get an error message too long and look at the
file ...
My intended behavior is to read the pb-message one at a time, analyze
its content and than read next message. For some reason. when I
deseralize from iostream, pb is reading and deseralizing full content
of the file.
How can I make the deserialization to work online, one message at a
time from the IoStream.
My code is attached below.
Many Thanks, Amit
#include stdio.h
#include iostream
#include string
#include fstream
#include cstdlib
#include temp.pb.h
using namespace std;
#define MAX 5
void write()
{
fstream output(out.db, ios::out | ios::trunc | ios::binary);
for(int num = 0; num MAX; ++num)
{
Person p;
p.set_id(num);
p.SerializeToOstream(output);
}
}
void read()
{
fstream input(out.db, ios::in | ios::binary);
int num = 0;
for(; num MAX; ++num)
{
Person p;
p.ParseFromIstream(input);
}
cout endl num endl;
}
int main()
{
write();
//cout Done Writing;
read();
cout Done reading endl;
return 0;
}
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: speed - python implementation
[+petar] On Mon, Oct 27, 2008 at 10:47 AM, andres [EMAIL PROTECTED] wrote: Hi, I would like to use protocol buffers in my python code but currently the serialization and parsing methods are too slow compared to cPickle. I've read several posts stating that this is because the python implementation has not been optimized for speed yet. Are there plans to improve the performance of proto buffers in python? Does anybody know of a C++ extension/wrapper module which lets you access C+ + compiled protocol buffers directly from python code? Thanks, Andres --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: protoc and python imports
I'm not sure I understand. What would you expect the import line importing a_pb2 to look like? My understanding is that Python imports are absolute, not relative to the importing file. On Sat, Oct 25, 2008 at 7:11 PM, Alan Kligman [EMAIL PROTECTED]wrote: I'm having a problem with protoc where python imports are not done correctly. Here's the situation: I have a directory structure like this: proto/a.proto proto/a/b.proto proto/a/c.proto a.proto provides some common definitions for both b.proto and c.proto. I build the output like this: protoc --proto_path=. --python_out=../dist *.proto protoc --proto_path=. --python_out=../dist a/*.proto assuming that proto is the current directory. Because a.proto is included in both b.proto and c.proto, they both import it like this: import a.proto;# relative to the current directory After building the protobuf files with protoc, the resulting python output has import statements for a_pb2.py that look like: import a_pb2.py which is wrong, because a_pb2.py is actually in the directory one above b_pb2.py and c_pb2.py. Is there a way to get protoc to do this properly? Is it a bug? Python2.5 handles relative imports, but there is no nice way to do it in python2.4. Thoughts? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Standard for RPC proto
I don't really have a stake in the design of a protobuf-based RPC format. However, I'd like to point out that the design philosophy we tend to prefer at Google is to keep each layer of the system as simple as possible, and implement orthogonal features using separate layers. Authentication is a great example of something that I would not want to make part of an RPC protocol itself, but rather implement as a layer under it, similar to the way HTTP can operate over SSL. If you keep the system separate in this way, it's much easier for people to avoid the overhead of features they don't need, find alternative ways of implementing individual features, and to reuse code in general. Just my opinion. 2008/10/27 Paul P. Komkoff Jr [EMAIL PROTECTED] On 26 окт, 02:53, Alan Kligman [EMAIL PROTECTED] wrote: I haven't had much to add recently. Protobuf-rpc is based heavily on json-rpc, so there's really nothing new behind it. It works well for my own use and is generic enough to probably work well for most other people. Is there a great deal of interest in devising a standard rpc protocol definition? Yes it is. Since everything is trying to design its own RPC format, running into the same flaws as everyone else. For example, I haven't seen (in protobuf-rpc neither in protorcp) a single word about authentification. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: protoc and python imports
The model used by the protocol compiler is to assume that the .proto files are located in a tree that parallels the Python package tree. We don't want to get into relative imports because they can get complicated and error-prone. If you don't want to put your .proto files into a tree matching your Python package tree, you could alternatively map them into such a tree virtually like so: protoc --proto_path=mypkg=proto This maps the virtual directory mypkg to the physical directory proto. You would then have to write your imports like: import mypkg/a.proto You can also map individual files. If this is insufficient then I guess we need a way to specify the python package explicitly in the .proto file, similar to the java_package option, rather than just inferring it from the location of the .proto file. On Tue, Oct 28, 2008 at 7:49 AM, Alan Kligman [EMAIL PROTECTED]wrote: I need the line to look like: from .. import a_pb2.py The reason this is a problem is because I'm building the protocol buffers into the middle of an existing project. The problem is that protoc assumes that the output is either at the top of the package, or that the related files are all in the same sub-package (which is rarely true). Python2.5 supports relative intra-package imports (like the one above). More details here: http://docs.python.org/tut/node8.html#SECTION00842. I think this is probably worth fixing. The workaround is to do some post-processing on the output from protoc, which could get nasty. On Oct 27, 5:44 pm, Kenton Varda [EMAIL PROTECTED] wrote: I'm not sure I understand. What would you expect the import line importing a_pb2 to look like? My understanding is that Python imports are absolute, not relative to the importing file. On Sat, Oct 25, 2008 at 7:11 PM, Alan Kligman [EMAIL PROTECTED] wrote: I'm having a problem with protoc where python imports are not done correctly. Here's the situation: I have a directory structure like this: proto/a.proto proto/a/b.proto proto/a/c.proto a.proto provides some common definitions for both b.proto and c.proto. I build the output like this: protoc --proto_path=. --python_out=../dist *.proto protoc --proto_path=. --python_out=../dist a/*.proto assuming that proto is the current directory. Because a.proto is included in both b.proto and c.proto, they both import it like this: import a.proto;# relative to the current directory After building the protobuf files with protoc, the resulting python output has import statements for a_pb2.py that look like: import a_pb2.py which is wrong, because a_pb2.py is actually in the directory one above b_pb2.py and c_pb2.py. Is there a way to get protoc to do this properly? Is it a bug? Python2.5 handles relative imports, but there is no nice way to do it in python2.4. Thoughts? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: protoc / license
The protoc code is licensed under the New BSD license, which is extremely permissive. You will have to include a copyright notice but that's about it. Read COPYING.txt for the full license -- it's quite short. On Fri, Oct 31, 2008 at 2:30 PM, Marc Gravell [EMAIL PROTECTED]wrote: (probably one for Kenton...) My current code-generation strategy requires the compiled binary form of a descriptor - i.e. by running the .proto through protoc. To make this more convenient, I'd like to include protoc in the archive, and have my code-generator call protoc automatically (so the caller only has to worry about .proto files). Are there any license issues doing this? Cheers, Marc --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Best way to define matrix4x4
The version with a single repeated field (which presumably you expect to
always have size 16) will be slightly more efficient on the wire and
significantly more efficient in-memory (assuming you're using C++).
I think you'll find the single repeated field version more usable, too --
you can actually iterate through the contents rather than having to identify
the all by name.
On Fri, Oct 31, 2008 at 1:51 PM, ketan [EMAIL PROTECTED] wrote:
Hi,
Which would be more compact representation for matrix:
message MatrixR
{
repeated double mat; // has to ensure that it passes values
correctly etc.
}
OR
message RowVector
{
required double rx = 1;
required double ry = 2;
required double rz = 3;
required double rw = 4;
}
message Matrix
{
required RowVector r0 = 1;
required RowVector r1 = 2;
required RowVector r2 = 3;
required RowVector r3 = 4;
}
or any other efficient method?
thx
Ketan
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: separate files per each message type
You would have to define each message in a separate .proto file. On Thu, Nov 6, 2008 at 1:37 AM, [EMAIL PROTECTED] wrote: Hi guys! I'm a new user. Can somebody kindly tell me how do I generate classes in separate files (h' and cpp' files for each message)? Thanks! Omer --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Interface and Implementation
Hi Jeff, I've read your message a few times now and I have to admit I don't really understand what you're getting at. Can you give a small example of each of the approaches you're considering? On Wed, Nov 5, 2008 at 12:06 AM, codeazure [EMAIL PROTECTED] wrote: Does anyone have any thoughts on the use of PB message definitions for interface only or throughout the implementation code as well? I am planning a very modular application, where each module uses PB as it's interface to external applications and inter-machine communications within itself. Should I be only writing PB definitions, generating C++ files and only using those? Or should I write C++ headers with the implementation version and only use PB for the interface? I can easily imagine using a PB generated header file in my implementation, calling the accessor functions taking advantage of the other support features in the Message class. But I'm not sure if this is a good approach. I understand that function calls are a different question I should only use RPC definitions when I am actually connecting to a remote application, and not for internal function calls. To put it another way, should I use the Bridge (or possibly Facade) design pattern when using PB or should my data structures be defined in one place only? Thanks, Jeff --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Sparc 64-bit bug fixed
I just submitted revision 72, which fixes the bug that caused the tests to crash on 64-bit sparc machines. It turns out DynamicMessage computed some byte offsets incorrectly leading to alignment problems. The new code should be more robust. I'll probably do a 2.0.3 release within the next week or two to get all these bug fixes out there. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Python for Symbian
The Python implementation currently does not require any non-Python libraries at runtime. You'll probably have to try it to see if it works on Symbian, though. :) On Fri, Nov 7, 2008 at 5:15 AM, Maximilian [EMAIL PROTECTED]wrote: Hi, Is it possible, to use Protobuf for python on Python for Symbian? http://sourceforge.net/projects/pys60/ Do I need some compiled libraries on the executing terminal, or is it (just for using the python code, no compilation) enough to have only the python code there in place? Best regards, Maximilian --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: [PATCH] Fix super().__init__ call on python2.6
I don't know Python very well. Petar, can you verify this is correct? (I tried it and the tests still pass.) On Thu, Nov 6, 2008 at 11:43 PM, Pavel Shramov [EMAIL PROTECTED] wrote: In 2.4, 2.5 and 2.6 superclass constructor call don't need self as first arg but 2.6 raises error if it's given. --- python/google/protobuf/reflection.py |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/python/google/protobuf/reflection.py b/python/google/protobuf/reflection.py index d80942e..25dd04b 100755 --- a/python/google/protobuf/reflection.py +++ b/python/google/protobuf/reflection.py @@ -152,7 +152,7 @@ class GeneratedProtocolMessageType(type): _AddMessageMethods(descriptor, cls) _AddPrivateHelperMethods(cls) superclass = super(GeneratedProtocolMessageType, cls) -superclass.__init__(cls, name, bases, dictionary) +superclass.__init__(name, bases, dictionary) # Stateless helpers for GeneratedProtocolMessageType below. -- 1.5.6.5 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: [PATCH] Fix super().__init__ call on python2.6
Yes, I believe you. We just have a policy of reviewing all changes. :) On Fri, Nov 7, 2008 at 11:19 AM, Pavel Shramov [EMAIL PROTECTED] wrote: On Fri, Nov 07, 2008 at 10:04:27AM -0800, Kenton Varda wrote: I don't know Python very well. Petar, can you verify this is correct? (I tried it and the tests still pass.) Problem is raising both in tests and in real applications. Both svn trunk and 2.0.2 version fail (will be strange if one won't) If it's interesting, system was FreeBSD 7.0 in KVM Test log included in the end of the message (it's pretty straightforward): Pavel Test log: [EMAIL PROTECTED] ~/protobuf/python]$ python2.6 ./setup.py test running test running egg_info writing protobuf.egg-info/PKG-INFO writing namespace_packages to protobuf.egg-info/namespace_packages.txt writing top-level names to protobuf.egg-info/top_level.txt writing dependency_links to protobuf.egg-info/dependency_links.txt reading manifest file 'protobuf.egg-info/SOURCES.txt' writing manifest file 'protobuf.egg-info/SOURCES.txt' running build_ext Traceback (most recent call last): File ./setup.py, line 131, in module Protocol Buffers are Google's data interchange format., File /usr/local/lib/python2.6/distutils/core.py, line 152, in setup dist.run_commands() File /usr/local/lib/python2.6/distutils/dist.py, line 975, in run_commands self.run_command(cmd) File /usr/local/lib/python2.6/distutils/dist.py, line 995, in run_command cmd_obj.run() File build/bdist.freebsd-7.0-RELEASE-i386/egg/setuptools/command/test.py, line 121, in run File build/bdist.freebsd-7.0-RELEASE-i386/egg/setuptools/command/test.py, line 101, in with_project_on_sys_path File build/bdist.freebsd-7.0-RELEASE-i386/egg/setuptools/command/test.py, line 130, in run_tests File /usr/local/lib/python2.6/unittest.py, line 816, in __init__ self.parseArgs(argv) File /usr/local/lib/python2.6/unittest.py, line 843, in parseArgs self.createTests() File /usr/local/lib/python2.6/unittest.py, line 849, in createTests self.module) File /usr/local/lib/python2.6/unittest.py, line 613, in loadTestsFromNames suites = [self.loadTestsFromName(name, module) for name in names] File /usr/local/lib/python2.6/unittest.py, line 598, in loadTestsFromName test = obj() File /usr/home/psha/protobuf/python/setup.py, line 62, in MakeTestSuite import google.protobuf.internal.generator_test as generator_test File /usr/home/psha/protobuf/python/google/protobuf/internal/generator_test.py, line 43, in module from google.protobuf import unittest_mset_pb2 File /usr/home/psha/protobuf/python/google/protobuf/unittest_mset_pb2.py, line 9, in module from google.protobuf import descriptor_pb2 File /usr/home/psha/protobuf/python/google/protobuf/descriptor_pb2.py, line 879, in module class FileDescriptorSet(message.Message): File /usr/home/psha/protobuf/python/google/protobuf/reflection.py, line 155, in __init__ superclass.__init__(cls, name, bases, dictionary) TypeError: Error when calling the metaclass bases type.__init__() takes 1 or 3 arguments --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Bug report and nearing completion hprotoc on par with protocol-buffers 2.0.2
On Mon, Nov 10, 2008 at 9:12 AM, Chris Kuklewicz [EMAIL PROTECTED]wrote: BUG * The user-defined options have the wrong value for some 32 bit value. Looks like you figured this out. Yeah, the idea is that 32-bit varints and 64-bit varints should always be compatible. So, if you write a 32-bit negative number as a varint, it needs to be sign-extended to 64 bits so that if it is read as a 64-bit varint you still get the correct result. The whole negative varints problem was a mistake made in an early version of protocol buffers that unfortunately we're stuck with now. BUG * The user-defined options from unittest_custom.proto have repetitions in the output from protoc that are not present in the .proto file. Not all fields are repeated (apparently just the fixed width ones), but this looks dangerous in the presence of repeated fields. Thanks, I'm looking into this. * The default_value of bytes and string types are stored differently. The bytes are stored in a raw form at the same escaping level as the proto file. A string is stored after the escape codes have been interpreted. ** Why, oh why, are they stored with different escape conventions? The default_value field of FileDescriptorProto is a string. Strings can only contain structurally-valid UTF-8 text. So, the default values for other strings can be represented just fine with no escaping, but raw bytes need to be escaped somehow such that they are valid UTF-8. In retrospect, this may not have been the best format. ** Is this documented anywhere? Yes, in the comments in descriptor.proto. * The name field of the FileDescriptorProto seems to be the file path passed on the command line or the filepath in the import statement. ** I have not checked, but if I were on windows would the file path from the command line have \ instead of / ? It will always be a forward slash. The path is actually not taken from the command line or from import statements. The path of each file is its location relative to the source tree defined by the --proto_path (or -I) flag. The goal here is to have a canonical name for every file. Note that this also implies that file names cannot contain . or .. components and cannot be absolute paths. ** Is this documented anywhere? I guess not as well as it should be. descriptor.proto describes the name field as file name, relative to the root of the source tree, but that's not precise enough for someone trying to write their own implementation. Sorry, my intent was never for people to write their own compiler; I hoped everyone would reuse libprotoc. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: C++: Optional Message problems
On Tue, Nov 11, 2008 at 7:46 AM, ChJees [EMAIL PROTECTED] wrote: I managed to solve my problem now. I used the blah.Bytesize() function instead of sizeof( blah.c_str() ) when sending the message size :3. Ah, yes, don't use c_str() with serialized protobufs because they may contain zeros before the end of the string. Either blah.ByteSize() or blah.size() would be valid things to use. Someone should really add to documentation about Sending and Receiving info between Server and Clients to prevent further Headaches. (Not small ones either!) I actually added a page yesterday covering some of these frequently asked questions: http://code.google.com/apis/protocolbuffers/docs/techniques.html --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Explanation of comment in descriptor.proto?
You don't have to worry about the uninterpreted_option field. It's an internal implementation detail of the parser. You should pretend it isn't there. The comment is saying that the uninterpreted_option field must have the name uninterpreted_option in all *Options messages because the parser uses reflection to access it. On Wed, Nov 12, 2008 at 2:47 AM, Jon Skeet [EMAIL PROTECTED] wrote: I was just looking over descriptor.proto for the option extensions, and I spotted this comment that I hadn't seen before: // Clients may define custom options as extensions of the *Options messages. // These extensions may not yet be known at parsing time, so the parser cannot // store the values in them. Instead it stores them in a field in the *Options // message called uninterpreted_option. This field must have the same name // across all *Options messages. We then use this field to populate the // extensions when we build a descriptor, at which point all protos have been // parsed and so all extensions are known. I don't understand the This field must have the same name across all *Options messages. Am I right in saying that's an implementation detail about the uninterpreted_option field? It's not saying that the extensions themselves need to have the same name across all *Options messages, right? Just thought it would be worth clarifying before I do the wrong thing :) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Is it possible to download the documentation?
Sorry, the docs are not actually stored in HTML form. They are EZT templates, so if we did make them downloadable you'd have to run an EZT web server to view them. The best we could do otherwise would be to spider them ourselves. So I don't think there's much worth doing here. On Wed, Nov 12, 2008 at 5:25 PM, Scott Stafford [EMAIL PROTECTED]wrote: Hi - My company works on an intranet that doesn't have an internet connection. For 3rd-party libraries, we typically download the documentation and host it locally. I plan to use protobuf, but I don't see anywhere to download the docs. Any chance they'll be added to the download package or offered as a separate package? I can spider them but it's a pain and all the URLs and CSS are screwy... Thanks, Scott --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: proto file naming and outer class collisions
BTW, you didn't have to rename your .proto file. You could have just used the java_outer_classname to make it produce a different class name. On Thu, Nov 13, 2008 at 11:40 AM, Kenton Varda [EMAIL PROTECTED] wrote: You're using the java_multiple_files option, I take it? The outer class is still necessary to store the file's descriptor and any top-level extension definitions. I suppose the descriptor could be moved into the first class defined in the file but it will take some work and seems like a low priority. I don't think it ever makes sense to put the extensions inside some other class but not all protos have top-level extensions. On Thu, Nov 13, 2008 at 10:04 AM, bmadigan [EMAIL PROTECTED] wrote: I think I'm missing somthing in the style guidelines. If I have a proto file called trip_summary.proto, which contains a message TripSummary, protoc throws the following error: -protoc-single: [exec] --java_out: trip_summary.proto: Cannot generate Java output because the file's outer class name, TripSummary, matches the name of one of the types declared inside it. Please either rename the type or use the java_outer_classname option to specify a different outer class name for the .proto file So, renaming the file to 'trip_summary_protos.proto', protoc generates 2 Java files, one called TripSummaryProtos.java, containing a static TripSummaryProtos, which contains an inner TripSummary class. Setting the option 'java_multiple_files=true', I get 2 java files, with the TripSummary class referencing all of the static members of TripSummaryProtos. This doesn't cause any problems with using TripSummary class (that I know of), but why is the container class necessary? Should I just define all of my messages in one file and ignore the outer class in my code? -bmadigan --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: null values should be treated as no value
I agree, the setters should either throw NPE or should treat setFoo(null)
the same as clearField().
On Thu, Nov 13, 2008 at 7:39 AM, bivas [EMAIL PROTECTED] wrote:
Hi,
I'm using the Java output feature of protobuf for my app.
I found that protobuf doesn't handle null values as well as i would
expect it.
for example:
message MyMessage {
required int32 id = 1;
optional string code = 2;
}
in java:
MyMessage message = MyMessage.newBuilder().setId(3).setCode(null).build
();
message.writeTo(System.out); // --- this will result a
NullPointerException when trying to write the code field
the reason is that in the created java source the setCode() method
looks like this:
public Builder setCode(java.lang.String value) {
result.hasCode = true; // the field flag indicate a value
while there isn't!
result.code_ = value;
return this;
}
method should look like:
// for optional fields
public Builder setCode(java.lang.String value) {
result.hasCode = (value != null) ;
result.code_ = value;
return this;
}
// for required fields
public Builder setCode(java.lang.String value) {
assert value != null;
result.hasCode = true;
result.code_ = value;
return this;
}
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: Custom EnumValueOptions?
Good news: There was a patch for this submitted to the internal version of
the code which should make its way to SVN today or tomorrow, and will be in
the 2.0.3 release.
On Wed, Nov 19, 2008 at 1:29 AM, Jerry Cattell [EMAIL PROTECTED]wrote:
I'm trying to use a custom EnumValueOption:
package test;
import google/protobuf/descriptor.proto;
extend google.protobuf.EnumValueOptions {
optional string code = 1234;
}
enum ProductType {
ROCK = 0 [(code) = R];
PAPER = 1 [(code) = P];
SCISSORS = 2 [(code) = S];
}
However, this doesn't seem to parse:
test.proto:9:12: Expected ;.
test.proto:10:13: Expected ;.
test.proto:11:16: Expected ;.
Looking at unittest_custom_options.proto, it mentions a TODO for
this. Are custom EnumValueOptions currently supported? If not, any
idea when they will be?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: RPC Design Structure
RpcController objects are per-request, not per-server or per-service. For every RPC request you make, you should have another RpcController object (though you can reuse an object by calling Clear() as long as you aren't making two requests at once). RpcChannel objects are per-service. Is that what you were thinking of? A single RpcChannel represents a connection to a single Service. However, there's nothing stopping you from multiplexing multiple RpcChannels across a single TCP connection, or creating a protocol that allows you to choose between multiple services exported by a server when constructing an RpcChannel. On Wed, Nov 19, 2008 at 5:06 PM, codeazure [EMAIL PROTECTED] wrote: On Nov 20, 7:58 am, Kenton Varda [EMAIL PROTECTED] wrote: I'm not sure I understand. There's nothing stopping you from spreading your definitions out among multiple .proto files which import each other, and there's nothing stopping you from exporting multiple services from a single server. You'll need to design a protocol that allows it, but protocol buffers doesn't do anything to prevent it. Can you be more specific about the problem you're facing, maybe giving an example? I can have multiple services per server? That's what I wanted to know. I could not any place in the documentation where it talked about whether Service has a 1:1 or Many:1 relationship with RpcController. It seemed like it was 1:1 from all the examples I had seen, hence my confusion. Next time you're updating the docs in this area, it might be worth changing the example to show 2 Services so it's clear to others. I was also thinking of services running on ports on Linux systems. In this situation, it is a 1:1 relationship between services and ports. I know it's a different situation, but my thinking got stuck on the term service. If I can define as many services as I like and attach them all to the same RPC Controller, then that will answer my question give me the extensible modular design I want. I had thought there had to be an answer like this because I couldn't imagine Google's internal applications would want to be bound by such a serious limitation. Thanks! --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: protocol buffer - compilation fails
This is a bug that has been fixed in SVN and will be fixed in the 2.0.3
release, which should happen soonish. If you don't use DynamicMessage you
don't need to worry about it.
On Thu, Nov 20, 2008 at 11:34 AM, Nimisha Mishra
[EMAIL PROTECTED] wrote:
Hello,
I have installed protocol buffer 2.0.2 version and trying to build it in
solaris environment. I get this error when I do a make check, Please note
– make ran fine. Please advise .
Making check in src
make check-am
make check-TESTS
Running main() from gtest_main.cc
[==] Running 557 tests from 85 test cases.
[--] Global test environment set-up.
[--] 1 test from GeneratedDescriptorTest
[ RUN ] GeneratedDescriptorTest.IdenticalDescriptors
[ OK ] GeneratedDescriptorTest.IdenticalDescriptors
[--] 22 tests from GeneratedMessageTest
[ RUN ] GeneratedMessageTest.Defaults
[ OK ] GeneratedMessageTest.Defaults
[ RUN ] GeneratedMessageTest.Accessors
[ OK ] GeneratedMessageTest.Accessors
[ RUN ] GeneratedMessageTest.MutableStringDefault
[ OK ] GeneratedMessageTest.MutableStringDefault
[ RUN ] GeneratedMessageTest.Clear
[ OK ] GeneratedMessageTest.Clear
[ RUN ] GeneratedMessageTest.EmbeddedNullsInBytesCharStar
[ OK ] GeneratedMessageTest.EmbeddedNullsInBytesCharStar
[ RUN ] GeneratedMessageTest.ClearOneField
[ OK ] GeneratedMessageTest.ClearOneField
[ RUN ] GeneratedMessageTest.CopyFrom
[ OK ] GeneratedMessageTest.CopyFrom
[ RUN ] GeneratedMessageTest.CopyConstructor
[ OK ] GeneratedMessageTest.CopyConstructor
[ RUN ] GeneratedMessageTest.CopyAssignmentOperator
[ OK ] GeneratedMessageTest.CopyAssignmentOperator
[ RUN ] GeneratedMessageTest.UpcastCopyFrom
[ OK ] GeneratedMessageTest.UpcastCopyFrom
[ RUN ] GeneratedMessageTest.DynamicMessageCopyFrom
bash: line 4: 15528 Bus Error (core dumped) ${dir}$tst
FAIL: protobuf-test
==
1 of 1 tests failed
Please report to protobuf@googlegroups.com
==
*** Error code 1
The following command caused the error:
failed=0; all=0; xfail=0; xpass=0; skip=0; \
srcdir=.; export srcdir; \
list='protobuf-test'; \
if test -n $list; then \
for tst in $list; do \
if test -f ./$tst; then dir=./; \
elif test -f $tst; then dir=; \
else dir=./; fi; \
if ${dir}$tst; then \
all=`expr $all + 1`; \
casein \
* $tst *) \
xpass=`expr $xpass + 1`; \
failed=`expr $failed + 1`; \
echo XPASS: $tst; \
;; \
*) \
echo PASS: $tst; \
;; \
esac; \
elif test $? -ne 77; then \
all=`expr $all + 1`; \
casein \
* $tst *) \
xfail=`expr $xfail + 1`; \
echo XFAIL: $tst; \
;; \
*) \
failed=`expr $failed + 1`; \
echo FAIL: $tst; \
;; \
esac; \
else \
skip=`expr $skip + 1`; \
echo SKIP: $tst; \
fi; \
done; \
if test $failed -eq 0; then \
if test $xfail -eq 0; then \
banner=All $all tests passed; \
else \
banner=All $all tests behaved as expected ($xfail expected
failures); \
fi; \
else \
if test $xpass -eq 0; then \
banner=$failed of $all tests failed; \
else \
banner=$failed of $all tests did not behave as expected ($xpass
unexpected passes); \
fi; \
fi; \
dashes=$banner; \
skipped=; \
if test $skip -ne 0; then \
skipped=($skip tests were not run); \
test `echo $skipped | wc -c` -le `echo $banner | wc -c` || \
dashes=$skipped; \
fi; \
report=; \
if test $failed -ne 0 test -n protobuf@googlegroups.com; then \
report=Please report to protobuf@googlegroups.com; \
test `echo $report | wc -c` -le `echo $banner | wc -c` || \
dashes=$report; \
fi; \
dashes=`echo $dashes | sed s/./=/g`; \
echo $dashes; \
echo $banner; \
test -z $skipped || echo $skipped; \
test -z $report || echo $report; \
echo $dashes; \
test $failed -eq 0; \
else :; fi
make: Fatal error: Command failed for target `check-TESTS'
Current working directory
/var/home/nmishra/protocol_buffer/protobuf-2.0.2/src
*** Error code 1
make: Fatal error: Command failed for target `check-am'
Current working directory
/var/home/nmishra/protocol_buffer/protobuf-2.0.2/src
*** Error code 1
make: Fatal error: Command failed for target `check'
Current working directory
/var/home/nmishra/protocol_buffer/protobuf-2.0.2/src
*** Error code 1
The following command caused the error:
failcom='exit 1'; \
for f in x $MAKEFLAGS; do \
case $f in \
*=* | --[!k]*);; \
*k*) failcom='fail=yes';; \
esac; \
done; \
Update roll-up submitted
I just committed SVN revision 76 which integrates a bunch of changes from our internal codebase (see below). 2.0.3 will probably be released next week. protoc * Enum values may now have custom options, using syntax similar to field options. * Fixed bug where .proto files which use custom options but don't actually define them (i.e. they import another .proto file defining the options) had to explicitly import descriptor.proto. * Adjacent string literals in .proto files will now be concatenated, like in C. C++ * Generated message classes now have a Swap() method which efficiently swaps the contents of two objects. * All message classes now have a SpaceUsed() method which returns an estimate of the number of bytes of allocated memory currently owned by the object. This is particularly useful when you are reusing a single message object to improve performance but want to make sure it doesn't bloat up too large. * New method Message::SerializeAsString() returns a string containing the serialized data. May be more convenient than calling SerializeToString(string*). * In debug mode, log error messages when string-type fields are found to contain bytes that are not valid UTF-8. * Fixed bug where a message with multiple extension ranges couldn't parse extensions. * Fixed bug where MergeFrom(const Message) didn't do anything if invoked on a message that contained no fields (but possibly contained extensions). * Fixed ShortDebugString() to not be O(n^2). Durr. * Fixed crash in TextFormat parsing if the first token in the input caused a tokenization error. Java * New overload of mergeFrom() which parses a slice of a byte array instead of the whole thing. * New method ByteString.asReadOnlyByteBuffer() does what it sounds like. * Improved performance of isInitialized() when optimizing for code size. Python * Corrected ListFields() signature in Message base class to match what subclasses actually implement. * Some minor refactoring. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
2.0.3rc1
I just put a release candidate here: http://groups.google.com/group/protobuf/files Test and tell me what's broken. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Add another option to support java_implement_interface
On Mon, Dec 1, 2008 at 6:46 PM, aantono [EMAIL PROTECTED] wrote:
message Person
java_implement_interface = com.domain.Name;
required string name;
required int32 age;
}
More precisely:
message Person {
option java_implement_interface = com.domain.Name;
required string name = 1;
required int32 age = 2;
}
And the effect of this is simply that when protoc generates the Person
class, it declares it as implementing com.domain.Name.
This is a very simple change, so I may be convinced to accept it. My main
concerns are:
* Adding another option imposes non-zero overhead even on people who don't
use the option, since it increases the code generated for descriptor.proto.
The size of descriptor.proto's generated code is already somewhat of a
problem.
* I'm somewhat uneasy with the idea of generated protobuf code depending on
non-generated code (other than standard libraries and libprotobuf). This
goes against the model. It would not work in our internal build system,
which assumes that protobuf code never has such dependencies. I am worried
that making the model more complicated in this way will lead to other
problems down the road; e.g. if protocol buffers are integrated better into
other build systems, they may also dislike this.
I can see where this feature would be useful but I'm not sure if it
outweighs these issues.
What do other people think?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: Slicing support in Python
Ehhh... Reference counting is slow (assuming it needs to be thread-safe),
and I think even adding it as an option would add an excessive amount of
complication to the system.
On Wed, Dec 3, 2008 at 2:04 PM, Dave Bailey [EMAIL PROTECTED] wrote:
On Dec 3, 2:00 pm, Dave Bailey [EMAIL PROTECTED] wrote:
On Dec 2, 10:49 pm, Kenton Varda [EMAIL PROTECTED] wrote:
C++ compatibility matters because eventually we want to be able to
generate
Python code which just wraps C++ code for efficiency. C++ isn't
garbage
collected, so append() can't easily be implemented in this case without
having ownership problems. Slice assignment has the same problem.
Also note that even pure-python protocol buffers have a sort of
ownership
issue: Sub-messages all contain pointers back to their parents, so
that
when a sub-message is modified, the parent's cached size can be marked
dirty. (Also, singular sub-messages have to inform their parents when
the
first field within them is set, but that doesn't apply here.)
(Here is my post without all of the ridiculous formatting):
While you're on this topic, I ran into this ownership issue while
implementing the Perl/XS wrapper around the generated C++ code. I
think it is the same issue that would face the author of a Python or
Ruby C++ extension of the generated C++. I ended up having to new() a
copy of every message that I transferred from C++ to Perl or vice
versa. So, for example, a statement like
$team-member($i)-set_first_name('Dave');
won't have the same effect as (C++)
team.mutable_member(i)-set_first_name(Dave);
because $team-member($i) will generate a copy of the underlying C++
object, so that it can be managed by Perl's reference counting without
any concern as to whether or not the underlying C++ object has been
deleted because the containing message went out of scope.
Anyway, I thought it might be possible to allow for shared ownership
of a message object if there were a reference counted variant of
RepeatedPtrFieldT (something like RepeatedSharedPtrFieldT or
whatever), which would provide incref() and decref() methods such that
Perl and C++ could use the same underlying C++ objects in the
generated code. This would really help the performance of the Perl/XS
code if all of that copy construction could be avoided somehow. The C+
+ code generator would need an option that would instruct it to
generate RepeatedSharedPtrFieldT members (and incref and decref
calls, where appropriate) for repeated messages (instead of using the
default RepeatedPtrFieldT).
What do you think? Is something like this possible, even though it
would require a change to protobuf? It is an issue for all {Python,
Perl, Ruby, ...}/C++ extension wrappers for Protocol Buffers. I have
found that protobuf is a faster Perl data serialization mechanism that
the (generic) Storable module, but I think it can be even faster.
-dave
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: Slicing support in Python
It still adds a lot of complication. And I think most cases where people
start out thinking thread-safety won't be an issue, particularly with
reference counting, they later find out otherwise.
On Wed, Dec 3, 2008 at 3:22 PM, Dave Bailey [EMAIL PROTECTED] wrote:
What if thread safety wasn't an issue?
-dave
On Dec 3, 2:41 pm, Kenton Varda [EMAIL PROTECTED] wrote:
Ehhh... Reference counting is slow (assuming it needs to be
thread-safe),
and I think even adding it as an option would add an excessive amount of
complication to the system.
On Wed, Dec 3, 2008 at 2:04 PM, Dave Bailey [EMAIL PROTECTED] wrote:
On Dec 3, 2:00 pm, Dave Bailey [EMAIL PROTECTED] wrote:
On Dec 2, 10:49 pm, Kenton Varda [EMAIL PROTECTED] wrote:
C++ compatibility matters because eventually we want to be able to
generate
Python code which just wraps C++ code for efficiency. C++ isn't
garbage
collected, so append() can't easily be implemented in this case
without
having ownership problems. Slice assignment has the same problem.
Also note that even pure-python protocol buffers have a sort of
ownership
issue: Sub-messages all contain pointers back to their parents, so
that
when a sub-message is modified, the parent's cached size can be
marked
dirty. (Also, singular sub-messages have to inform their parents
when
the
first field within them is set, but that doesn't apply here.)
(Here is my post without all of the ridiculous formatting):
While you're on this topic, I ran into this ownership issue while
implementing the Perl/XS wrapper around the generated C++ code. I
think it is the same issue that would face the author of a Python or
Ruby C++ extension of the generated C++. I ended up having to new() a
copy of every message that I transferred from C++ to Perl or vice
versa. So, for example, a statement like
$team-member($i)-set_first_name('Dave');
won't have the same effect as (C++)
team.mutable_member(i)-set_first_name(Dave);
because $team-member($i) will generate a copy of the underlying C++
object, so that it can be managed by Perl's reference counting without
any concern as to whether or not the underlying C++ object has been
deleted because the containing message went out of scope.
Anyway, I thought it might be possible to allow for shared ownership
of a message object if there were a reference counted variant of
RepeatedPtrFieldT (something like RepeatedSharedPtrFieldT or
whatever), which would provide incref() and decref() methods such that
Perl and C++ could use the same underlying C++ objects in the
generated code. This would really help the performance of the Perl/XS
code if all of that copy construction could be avoided somehow. The C+
+ code generator would need an option that would instruct it to
generate RepeatedSharedPtrFieldT members (and incref and decref
calls, where appropriate) for repeated messages (instead of using the
default RepeatedPtrFieldT).
What do you think? Is something like this possible, even though it
would require a change to protobuf? It is an issue for all {Python,
Perl, Ruby, ...}/C++ extension wrappers for Protocol Buffers. I have
found that protobuf is a faster Perl data serialization mechanism that
the (generic) Storable module, but I think it can be even faster.
-dave
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: [Fwd: Re: Streaming]
It's quite easy to write a helper function that reads/writes delimited
messages (delimited by size or by end tag).
For example, here's one for writing a length-delimited message:
bool WriteMessage(const Message message, ZeroCopyOutputStream* output) {
CodedOutputStream coded_out(output);
return coded_out.WriteVarint32(message.ByteSize())
message.SerializeWithCachedSizes(coded_out);
}
and here's one for reading one message:
bool ReadMessage(ZeroCopyInputStream* input, Message* message) {
CodedInputStream coded_in(input);
uint32 size;
if (!coded_in.ReadVarint32(size)) return false;
CodedInputStream::Limit limit = coded_in.PushLimit(size);
if (!message-ParseFromCodedStream(coded_in)) return false;
if (!coded_in.ExpectAtEnd()) return false;
coded_in.PopLimit(limit);
return true;
}
(I haven't tested the above so it may contain minor errors.)
We could add these as methods of the Message class. Note, though, that for
many applications, this kind of streaming is too simplistic. For example,
the above will not allow you to efficiently seek to an arbitrary message in
the stream, since at the very least you have to read the sizes of all
messages before it to find it. It's also not very robust in the face of
data corruption -- if any of the sizes are corrupted, the whole stream is
unreadable. So, you may find you want to do something more complicated,
depending on your app. But, anything more complicated is really beyond the
scope of the protocol buffer library.
On Fri, Dec 5, 2008 at 8:27 AM, Shane Green [EMAIL PROTECTED]wrote:
Thanks very much Jon (see below). You make good points and I like the
approach the you describe. I am still thinking, however, that there is
power in the ability for message instances to write and parse themselves
from a stream.
A message instance could be passed a stream object which chains back to
the network connection from which bytes are being received. A stop flag
based parsing mechanism could be passed this buffer object, and would
handle reading the stream and initializing its properties, exiting when
the serialization of that message instance stopped. At this point, a
new message instance could be created, and the process repeated.
The type of message doing the parsing could vary from message to
message, even with the serializations being sent and received back to
back. This mechanism would work regardless of field-types being
streamed. A message type consisting solely of varint fields, whose
length is determined while reading the varint's value, would support
streaming no differently than any other message type.
The solution also seems to support every requirement supported by the
original buffer type. Messages serialized to a buffer, could just as
easily be initialized from that buffer as they could from the string
contained by the buffer.
m1 = Message()
buffer = Buffer()
[...] (initialize instance vars)
m1.SerializeToBuffer(buffer)
m2 = Message()
m2.ParseFromBuffer(buffer)
Produces same result as:
m2 = Message()
bytes = m1.SerializeToString()
m2.ParseFromString(bytes)
The string-based parse would ignore the stop bit, parsing the entire
string. The buffer-based parsing would stop parsing when the stop bit,
producing the same result.
Handling of concatenated serializations is supported through repeated
calls to parse from buffer:
m1 = Message()
[...] (initialize instance vars)
m2 = Message()
[...] (initialize instance vars)
buffer = Buffer()
m1.SerializeToBuffer(buffer)
m2.SerializeToBuffer(buffer)
m3 = Message()
m3.ParseFromBuffer(buffer)
m3.ParseFromBuffer(buffer)
Would produce same result as:
m3 = Message()
m3.ParseFromString(m1.SerializeToString() + m2.SerializeToString())
As long as an unused, and never to be used, field number is used to
generate the stop bit's key, then I don't believe there are any
incompatibilities between buffer-based message marshalling and the
existing string-based code. A very easy usage:
# Sending side
for message in messages:
message.SerializeToBuffer(buffer)
# Receiving side
for msgtype in types:
message = msgtype()
message.ParseFromBuffer(buffer)
Unless I've overlooked something, it seems like the stream based
marshalling and unmarshalling is powerful, simple, and completely
compatible with all existing code. But there is a very real chance I've
overlooked something...
- Shane
Forwarded Message
From: Jon Skeet [EMAIL PROTECTED]
To: Shane Green [EMAIL PROTECTED]
Subject: Re: Streaming
Date: Fri, 5 Dec 2008 08:19:41 +
2008/12/5 Shane Green [EMAIL PROTECTED]
Thanks Jon. Those are good points. I rather liked the
self-delimiting
nature of fields, and thought this method would bring that
feature up to
the message level, without breaking any of the existing
capabilities.
So my goal was a message which could truly
Re: Slicing support in Python
On Fri, Dec 5, 2008 at 10:59 PM, Alek Storm [EMAIL PROTECTED] wrote: On Wed, Dec 3, 2008 at 5:32 AM, Kenton Varda [EMAIL PROTECTED] wrote: Sorry, I think you misunderstood. The C++ parsers generated by protoc (with optimize_for = SPEED) are an order of magnitude faster than the dynamic *C++* parser (used with optimize_for = CODE_SIZE and DynamicMessage). The Python parser is considerably slower than either of them, but that's beside the point. Your decoupled parser which produces a tag/value tree will be at least as slow as the existing C++ dynamic parser, probably slower (since it sounds like it would use some sort of dictionary structure rather than flat classes/structs). Oh, I forgot we have two C++ parsers. The method I described uses the generated (SPEED) parser, so it should be a great deal quicker. It just outputs a tree instead of a message, leaving the smart object creation to Python. No, the static (SPEED) parser parses to generated C++ objects. It doesn't make sense to say that we'll use the static parser to parse to this abstract tree structure, because the whole point of the static parser is that it parses to concrete objects. If it didn't, it wouldn't be so fast. (In fact, the biggest bottleneck in protobuf parsing is memory bandwidth, and I can't see how your tree structure would be anywhere near as compact as a generated message class.) But then, protobuf objects might as well be your tree. There's no reason to define a separate tree structure when we already have a structure explicitly designed for holding protocol buffer data. Your tree probably would not be any easier to access than an actual protocol message object. Honestly, I think using reflection for something as basic as changing the ouput format is hackish and could get ugly. I think you're thinking of a different kind of reflection. I'm talking about the google::protobuf::Reflection interface. The whole point of this interface is to allow you to do things like write custom codecs for things like JSON or XML. Take a look at text_format.cc for an example usage. But just for the record, I'm pretty sure Python's list remove() method compares by value, and doesn't have a method that compares by identity. So there would be no reason to include a compare-by-identity method in protobuf repeated fields. OK, well, honestly, I think a remove-by-value makes even less sense. Comparing large non-flat data structures by value is awkward and rarely useful. But this is exactly the kind of thing that we don't know if we don't have a use case to examine. Okay, you place more value on compact interface. So are we keeping remove() for scalar values? I think their interfaces should be consistent, but I don't think you think that's as important. remove() makes sense for scalars. There's no question of identity vs. value comparison, and it's not awkward or unusual to compare two scalar values. Okay. So let's say we have a pure-C++ parser with a Python wrapper. This brings us back to getting slicing to work in C++ with no garbage collector. Kenton, could you elaborate on what you meant earlier by ownership problems specific to the C++ version? I can't really see anything that would affect PB repeated fields that isn't taken care of by handing the user control over allocation and deallocation of the field elements. If a message Foo has a repeated field of type Bar, then the Bar objects in that field are owned by Foo. When you delete Foo, all the Bars are deleted. Leaving it up to the user to delete the Bar objects themselves is way too much of a burden. Is there anything wrong with having a list of parents? I'm guessing I'm being naive - would speed be affected too much by that? Way too complicated, probably a lot of overhead, and not very useful in practice. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: how to parse a file with millions of records with protobuf
On Sun, Dec 7, 2008 at 3:45 AM, nightwalker leo [EMAIL PROTECTED]wrote: when I try to parse an addressbook file which has 2^20 records of person , my program complains like this: libprotobuf WARNING D:\protobuf-2.0.2\src\google\protobuf\io \coded_stream.cc:459] Reading dangerously large protocol message. If the message turns out to be larger than 67108864 bytes, parsing will be halted for security reasons. To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/ protobuf/io/coded_stream.h. how to deal with the problem in an elegant way instead of increasing the limit or simply turning off the warning message? The documentation for SetTotalBytesLimit() answers your question: http://code.google.com/apis/protocolbuffers/docs/reference/cpp/google.protobuf.io.coded_stream.html#CodedInputStream.SetTotalBytesLimit.details --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Quick Hacky text_mode Parse in Python
Hey Petar, isn't there a patch someone was trying to submit that implements
text format parsing? (For real, not by wrapping protoc.) What's the status
of that?
On Mon, Dec 8, 2008 at 5:03 AM, Nicholas Reid [EMAIL PROTECTED] wrote:
Hi All,
Firstly, just wanted to thank Kenton and the Google team, PB2 is a
beautiful piece of work! Thanks heaps.
I will almost certainly go to some deep circle of Programmer's Hell
for this, but it might be useful for someone until the guys get a
chance to add text_mode message parsing functionality to the Python
API. There are almost certainly more elegant ways of doing this.
Code:
def parse_text_format(message_string, generated_message_type):
Parses the given Protobuf text_format into a new instance of
the given type.
# Should be defined globally somewhere
PROTO_FILENAME = person.proto
# Instance new message
obj = generated_message_type()
# Wrap the protoc command-line utility, expects that 'protoc'
should be on your PATH somewhere
(stdout, stdin) = popen2.popen2(protoc %s --encode=%s %
(PROTO_FILENAME, message_type.DESCRIPTOR.name), bufsize=1024)
# Feed in the message_string in text_format
stdin.write(message_string)
stdin.close()
# Read out the protoc-encoded binary format
binary_string = stdout.read()
stdout.close()
# Parse the resulting binary representation.
obj.ParseFromString(binary_string)
return obj
Example:
Assuming person.proto contains:
message Person {
required string name = 1;
}
Code:
from person_pb2 import *
guido = parse_text_format(name: Guido, person)
Should give you a person object which you can use for nefarious
purposes.
Kind regards,
Nicholas Reid
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: Slicing support in Python
On Sat, Dec 6, 2008 at 1:03 AM, Alek Storm [EMAIL PROTECTED] wrote: But it does give us a lot of cool functionality, like adding the same message to two parents, and (yes!) slicing support. I thought this was common practice in C++, but it's been quite a while since I've coded it. Nope, in the C++ world we have to worry excessively about ownership, and we generally make defensive copies rather than trying to allow an object to be referenced from two places. Is it really that useful to have ByteSize() cached for repeated fields? If it's not, we get everything I mentioned above for free. I'm genuinely not sure - it only comes up when serializing the message in wire_format.py. What do you think? Yes, it's just as necessary as it is with optional fields. The main problem is that the size of a message must be written before the message contents itself. If, while serializing, you call ByteSize() to get this size every time you write a message, then you'll end up computing the size of deeply-nested messages many times (once for each outer message within which they're nested). Caching avoids that problem. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: 2.0.3 officially released
Yes, thanks for reminding me. There still isn't any documentation on this feature, even inside Google. :( I will try to write it this week. (Sorry, it all comes down to me having too much work to do...) On Sun, Dec 14, 2008 at 4:44 PM, codeazure codeaz...@gmail.com wrote: The language guide in the documentation still doesn't describe Field Options. I can see some references to them in some of the language specific sections (Java, Python), but nothing telling us how to use this feature either in fields or enums. With this and other changes, it's really helpful for developers to have access to current and complete documentation - doesn't have to be a huge amount, but just something to remind us that it's there. Thanks, Jeff On Dec 6, 5:48 am, Kenton Varda ken...@google.com wrote: protoc * Enum values may now have custom options, using syntax similar to field options. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: How to migrate to protocol buffers from bean style data structure
On Tue, Dec 16, 2008 at 2:27 AM, atulatri2...@gmail.com wrote:
Hi,
I have couple of questions regarding protocol buffers in java
1. I am using old bean style data structures. How can i migrate to
protocol buffers and what problems may arise due to this migration.
This is a pretty broad question with many reasonable answers. You could
keep your existing structures and just add methods to convert between them
and protocol buffers (see below). This would be easy but would be annoying
to maintain since every time you added a field you'd have to update the bean
class. Alternatively, you could completely replace your bean-style classes
with protocol message classes, but this will require refactoring your code
to work with immutable objects. Most code tends to be pretty easy to
refactor (because most code will tend to construct data objects all at once
and then not modify them afterwards, which is what protocol buffer objects
require), but some code may be more complicated to rework. Also, if you
have a lot of code it could be painful. A third option is to replace your
bean objects with protocol message builders and use those everywhere, since
they have both setters and getters and thus can operate like mutable
objects. However, this is kind of ugly and not the way builders are meant
to be used.
I would probably recommend the first option, as it keeps your code more
independent of protocol buffers, so you can more easily change your mind
later if you want to.
2. How can i serialize bean style data structures using protocol
buffers?
You'd need to add methods which convert between your bean class and protocol
message objects. For example:
public MyMessageType toProto() {
return MyMessageType.newBuilder()
.setFoo(this.getFoo())
.setBar(this.getBar())
.setBaz(this.getBaz())
.build();
}
public void fromProto(MyMessageType proto) {
this.setFoo(proto.getFoo());
this.setBar(proto.getBar());
this.setBaz(proto.getBaz());
}
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: 2.0.3rc1
I think the problem may be that the script was written on unix, so it does not expect protoc to have the .exe extension. That's a bug that should be fixed. I'm surprised it hasn't been reported before. Does it work if you put protoc.exe somewhere in your PATH? On Wed, Dec 17, 2008 at 4:04 AM, 王鹏生 pengshengw...@gmail.com wrote: Hi, I download protoc-2.0.3-win32.zip and protobuf-2.0.3.zip from http://code.google.com/p/protobuf/downloads/list Unzip protobuf-2.0.3.zip to C:\Python25\protobuf-2.0.3 and put protoc.exe to C:\Python25\protobuf-2.0.3\src\ Then, I run the command: C:\Python25\protobuf-2.0.3\pythonsetup.py install But it told me that protoc is not installed nor found in ../src. Why? How? Thanks! I test the other version like protobuf-2.0.2 and protobuf-2.0.1 but it had the same problem... The detail message as below: C:\Python25\protobuf-2.0.3\pythonsetup.py install --- This script requires setuptools version 0.6c8 to run (even to display help). I will attempt to download it for you (from http://pypi.python.org/packages/2.5/s/setuptools/), but you may need to enable firewall access for this script first. I will start the download in 15 seconds. (Note: if this machine does not have network access, please obtain the file http://pypi.python.org/packages/2.5/s/setuptools/setuptools-0.6c8-py2.5.egg and place it in this directory before rerunning this script.) --- Downloading http://pypi.python.org/packages/2.5/s/setuptools/setuptools-0.6c8-py 2.5.egg Generating google/protobuf/descriptor_pb2.py... protoc is not installed nor found in ../src. Please compile it or install the b inary package. 2008/12/3 Kenton Varda ken...@google.com: You need to run setup.py from the directory where it resides. On Tue, Dec 2, 2008 at 11:51 PM, EYN pengshengw...@gmail.com wrote: I can not install it: C:\Python25python C:\Python25\protobuf-2.0.3rc1\python\setup.py install Can't find required file: ../src/google/protobuf/descriptor.proto On 11月26日, 上午10时32分, Kenton Varda ken...@google.com wrote: I just put a release candidate here:http://groups.google.com/group/protobuf/files Test and tell me what's broken. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Enum for field numbers
OK, I'll stick it on my TODO list. Feel free to send me a patch if you want it done faster. :) On Tue, Dec 16, 2008 at 10:53 PM, sorent soren.torstens...@tietoenator.comwrote: Definitely a better idea better :-) /Sören --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: C++ SerializeToArray
When you call ParseFromArray(), you must pass the exact size of the message
being parsed. You cannot simply give it the size of the buffer if there are
extra bytes after the end of the message.
What happened here is that the parser, after parsing the bytes of your
actual message, continued to interpret the following bytes as more data.
The next byte was a 1, which the parser thought was indicating the
beginning of a 64-bit fixed-width field with field number zero. Since your
message does not declare a field number zero (you actually aren't allowed
to), it treated this as an unknown field and stored the value in the
UnknownFieldSet. Later, when you serialized the message, the value in the
UnknownFieldSet was written out again.
The moral of the story is that you must transmit the size of your message
along with the data, so that you can pass it to ParseFromArray() on the
receiving end.
On Wed, Dec 17, 2008 at 2:22 PM, Ryan ryanroll...@gmail.com wrote:
Ok - found it. The problem was not on the serialization but on the
message creation/parsing. The buffer the C++ code was parsing the
protocol buffer message from was not properly initialized causing the
ByteSize() function to return an incorrect value based on the junk
padded past the typical sentinel.
I did a call to memset prior and all is good.
Two additional thoughts:
1) I am surprised the C++ parsing succeeded?
2) Why was the serialization reproducing the bad input? I would think
the process of Marshalling/Unmarshalling would of cleaned it up.
On Dec 17, 10:48 am, Ryan ryanroll...@gmail.com wrote:
I will try. A good debugging step for myself if not anything else.
This system is of course distributed so it may be difficult to isolate
it down into something I can send you easily (also lots of
dependencies). I have been trying to identify the messages that are
causing this but it appears to be happening independent of the message
contents.
In a particular sequence of messages the same messages always get the
erroneous bytes appended. However, if I take one of those messages and
send it independently - works fine.
Weird...
Thanks for the dialog.
On Dec 17, 9:36 am, Kenton Varda ken...@google.com wrote:
If you write a program demonstrating the problem and send it to me, I
can
debug it.
On Wed, Dec 17, 2008 at 11:10 AM, Ryan ryanroll...@gmail.com wrote:
char mess[MAX_MESSLEN] ;
.
//Load some properties for the group com call
.
.
bplMessage.SerializeToArray(mess,bplMessage.ByteSize());
//what I am using to detect trailing bytes
for(int i =0; i bplMessage.ByteSize();i++) {
std::cout (int) mess[i] std::endl;
}
On Dec 17, 8:32 am, Kenton Varda ken...@google.com wrote:
Hi Ryan,
What does your code look like that calls SerializeToArray()? It
should
be
something like:
int size = message.ByteSize();
char* array = new char[size];
message.SerializeToArray(array, size);
On Tue, Dec 16, 2008 at 8:13 PM, Ryan ryanroll...@gmail.com
wrote:
I have been working with a Group Communication System and
Protocol
Buffers.
I have an issue where the C++ SerializeToArray call on one of my
messages is occasionally appending Bytes {1,0,0,0,0,0,0,0,0} to
the
end of the returned character array?
Any ideas on what might be causing this? I can
Marshall/Unmarshall
fine using the java api but the C++ call above has the odd quirk
mentioned.
The Java parsingFrom fails on the C++ generated messages that
have the
above bytes appended.
Any suggestions much appreciated.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: Light in-memory use
Size-wise, the only difference between your Lite structure and the current
protobuf classes is that it wouldn't have a virtual table pointer or a
cached byte size. Do these really make that much difference?
On Wed, Dec 17, 2008 at 9:11 PM, Shaun Cox shaun8...@gmail.com wrote:
As stated by previous posts (the uint16 one in particular), protobuf C+
+ is not designed for extremely-memory-constrained systems. I was
curious about the decision to not make the bare bones structure of
the message public and provide a way to construct one from another.
For example right now the generated code looks something like this:
couch.proto:
message Couch
{
required uint32 width;
required uint32 length;
}
couch.pb.h:
class Couch : public ::google::protobuf::Message {
public:
Couch();
// rest of public interface...
private:
// pb data members
uint32 width_;
uint32 length_;
// more pb data member
};
If pb provided a bare bones structure it might look something like:
couch.pb.h:
class Couch : public ::google::protobuf::Message {
public:
struct Lite {
Lite(const Couch from);
uint32 width_;
uint32 length_;
};
Couch();
Couch(const Couch::Lite from);
// rest of public interface...
private:
// pb data members
Lite impl_;
// more pb data member
};
First off, the API for constructing one from another is more just to
show an example than anything else. Exposing the underlying structure
allows you to store smaller in-memory copies of the message. This
comes into play more when the structure is part of a larger class that
isn't going across the wire.
class LivingRoom {
Couch::Lite couches[10];
};
class Apartment{
LivingRoom rooms[3];
};
class Complex {
Apartment apartments[1];
};
class City {
Complex complexes[10];
}
It seems wasteful to have Couch when you could just as well have
Couch::Lite. With a change like this, if you aren't as concerned with
memory usage, there is no change to how pb behaves or how you use
it.
Shaun
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: NewCallback does not take 3 arguments
I can't reproduce this problem. What compiler are you using? Can you
provide example code which demonstrates the problem?
On Mon, Dec 22, 2008 at 2:18 AM, ufukalti...@gmail.com wrote:
Btw, i'm using the latest svn version of the compiler and library
On Dec 20, 11:28 pm, ufukalti...@gmail.com wrote:
Hello,
I'm trying to implement a socket based RPC implementation with
protobuf but i'm stuck at creating Closure.
// Example that calls a method:
// class Handler {
//public:
// ...
//
// void FooDone(const FooResponse* response) {
// ...
// }
//
// void CallFoo() {
// ...
// // When done, call FooDone() and pass it a pointer to the
response.
// Closure* callback = NewCallback(this, Handler::FooDone,
response);
// // Make the call.
// service-Foo(controller, request, response, callback);
// }
// };
I'm following this example but it gives an error complaining about:
function does not take 3 arguments: NewCallback
Can you please help me on this?
Thank you.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: 2.0.3 officially released
On Thu, Dec 25, 2008 at 6:10 PM, Chris turingt...@gmail.com wrote: There is a small type on the web page about the custom options: optional int32 foo = 1 [(my_field_option = 4.5)]; should be optional int32 foo = 1 [(my_field_option) = 4.5]; Thanks, I'll fix this. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: 2.0.3 officially released
On Sat, Jan 3, 2009 at 7:32 PM, Dave Wolfe dwolfe5...@hotmail.com wrote: FYI, it looks like a few new functions are missing __declspec() decorators. I had to apply the trivial patch below in order to build a DLL version of protobuf under Windows. (This applies to both 2.0.3 and the current trunk...) Ouch, I guess I should explicitly test compiling DLLs before releasing. Strange, though, because I have a script which is supposed to add the export decls for me. Anyway, I'll look into this; thanks. -- Index: src/google/protobuf/repeated_field.h === --- src/google/protobuf/repeated_field.h(revision 90) +++ src/google/protobuf/repeated_field.h(working copy) @@ -93,7 +93,7 @@ }; // We need this (from generated_message_reflection.cc). -int StringSpaceUsedExcludingSelf(const string str); +LIBPROTOBUF_EXPORT int StringSpaceUsedExcludingSelf(const string str); } // namespace internal Index: src/google/protobuf/stubs/common.h === --- src/google/protobuf/stubs/common.h (revision 90) +++ src/google/protobuf/stubs/common.h (working copy) @@ -1075,7 +1075,7 @@ // Checks if the buffer contains structurally-valid UTF-8. Implemented in // structurally_valid.cc. -bool IsStructurallyValidUTF8(const char* buf, int len); +LIBPROTOBUF_EXPORT bool IsStructurallyValidUTF8(const char* buf, int len); } // namespace internal Index: src/google/protobuf/generated_message_reflection.h === --- src/google/protobuf/generated_message_reflection.h (revision 90) +++ src/google/protobuf/generated_message_reflection.h (working copy) @@ -383,7 +383,7 @@ // Compute the space used by a string, not including sizeof(string) itself. // This is slightly complicated because small strings store their data within // the string object but large strings do not. -int StringSpaceUsedExcludingSelf(const string str); +LIBPROTOBUF_EXPORT int StringSpaceUsedExcludingSelf(const string str); } // namespace internal --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Build errors on installation
Did the tests pass or fail? The tests explicitly test the code which produces those warnings, so seeing them is expected. As long as the tests report that they passed you are fine. On Mon, Jan 5, 2009 at 12:12 PM, mikesparr...@gmail.com mikesparr...@gmail.com wrote: Starting a clean installation Running MS Visual Studio 2008(version 9.0.30729.1 SP) and following the readme.txt instructions I open the sln file and get a convert this solution message which appears to go ok. I build the solution in Debug mode I open a cmd window and run tests.exe and get the following errors/ warnings: libprotobuf ERROR c:\proj\protobuf-2.0.3\src\google\protobuf\io \coded_stream.cc: 170] A protocol message was rejected because it was too big (more than 16 bytes) . To increase the limit (or to disable these warnings), see CodedInputStream::S etTotalBytesLimit() in google/protobuf/io/coded_stream.h. libprotobuf WARNING c:\proj\protobuf-2.0.3\src\google\protobuf\stubs \strutil_uni ttest.cc:67] Couldn't set locale to es_ES. Skipping this test. libprotobuf ERROR c:\proj\protobuf-2.0.3\src\google\protobuf \wire_format_inl.h:2 89] Encountered string containing invalid UTF-8 data while serializing protocol buffer. Strings must contain only UTF-8; use the 'bytes' type for raw bytes. Thanks in advance, Mike --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Pb with online doc on website?
On Mon, Jan 5, 2009 at 3:08 PM, Alain M. ala...@pobox.com wrote: BTW, is there a users' list about Chrome? I'm sure there is, but I don't have any easier way of finding out than just Googling it. :) Alain Olivier Gautherot escreveu: Hi Kenton, thanks for the insight. On Mon, Jan 5, 2009 at 4:38 PM, Kenton Varda ken...@google.com mailto:ken...@google.com wrote: Hmm, works for me. This may sound silly, but try deleting all your google.com-related cookies. I've seen redirect loops in google services fixed by this before. As a matter of fact, IE and Firefox work but Chrome does not (it is my main browser, actually). I tried to clear all personal data but no difference. There seems to be something weird - but I have alternatives so it's not a showstopper. Thanks for the support Olivier On Fri, Jan 2, 2009 at 9:36 AM, oliv...@gautherot.net mailto:oliv...@gautherot.net wrote: Hi! I tried to access the online doc at http://code.google.com/apis/protocolbuffers/docs/overview.html but, for some reason, it reports a loop in the links. It seems to be a general problem as all the docs I tried to open reported the same. I know it used to work in the past. I guess someone tried to evaluate if somebody actually RTFM ;-) (Sorry, just a joke, please no flame ;-) ) Happy new year to you all and thanks for the great work! Cheers Olivier -- Olivier Gautherot --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Generating .proto files from C++ classes
How would that work, exactly? On Wed, Jan 7, 2009 at 11:50 AM, mikesparr...@gmail.com mikesparr...@gmail.com wrote: Is there an existing convert that can take a header file and produce a .proto file? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Generating .proto files from C++ classes
Protocol buffers are useful even if all users are using the same language. In fact, we used them in C++ for some time before anyone bothered writing Java and Python implementations. On Wed, Jan 7, 2009 at 3:43 PM, Alek Storm alek.st...@gmail.com wrote: Mike, what exactly are you trying to accomplish? Do you just want to persist a C++ class? If so, you don't need to generate a .proto file, because other languages won't need to read it, and you know your own encoding layout. So you could still use the Protocol Buffers wire format. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Generating .proto files from C++ classes
On Wed, Jan 7, 2009 at 5:39 PM, Alek Storm alek.st...@gmail.com wrote: Right, I guess I didn't explain why they don't work in this case. Generating .proto files from C++ headers is obviously the reverse of the normal sequence. In the PB method of versioning, the original .proto is written and deployed. The next version then writes an extension of the original. If the .proto file is generated from C+ + headers, every time the headers are changed, the .proto is wiped out and replaced with a new one. This is dangerous if a new C++ class version redefines a field - it would break old code. To sum up, you'd lose the benefits of the PB versioning system. You certainly could generate .proto files from C++, but I don't know what you'd use them for - if you wanted to generate, for example, Java bindings from them, then just write the .proto file manually in the first place and generate C++ and Java bindings from that. Yep, good points. That being said, you could still use the Protocol Buffers wire format, but you'd probably need to add a version tag to each message. You can use extend the Boost Serialization library for this, and Mike, I have a note or two on this if you're interested. IMO, there's not much reason to use the protobuf wire format unless you explicitly intend for some users to read/write the format using actual protocol buffers. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Return value from CallMethod, default callback
I believe there was a proposal internally that if the caller passes None for the callback, the RPC implementation should block until completion and then return the result. Does that seem like what you want? On Wed, Jan 7, 2009 at 6:23 PM, Alan Kligman alan.klig...@gmail.com wrote: I poked around a bit and the code above isn't correct (for a couple of reasons). But the question remains: should stubs be generated that return the response down the callstack instead of passing it through a callback function? On Jan 7, 5:59 pm, Alan Kligman alan.klig...@gmail.com wrote: Does it make sense to change stub.rpc_channel.CallMethod( method_descriptor, rpc_controller, request, method_descriptor.output_type._concrete_class, callback) to return stub.rpc_channel.CallMethod( method_descriptor, rpc_controller, request, method_descriptor.output_type._concrete_class, callback=lambda x:None) in python/google/protobuf/service_reflection.py? If, for example, I'm running in a single thread and my underlying channel is blocking, it's strange to pass a callback in when control would normally return to the calling method after CallMethod has completed synchronously. Thoughts? a --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Return value from CallMethod, default callback
Sounds fine to me, but it's up to Petar (cc'd). On Wed, Jan 7, 2009 at 7:04 PM, Alan Kligman alan.klig...@gmail.com wrote: Ya, that's exactly what I'm experimenting with now and precisely what I'm looking for. Would be perfect if the callback had a default value of None too. a On Jan 7, 10:02 pm, Kenton Varda ken...@google.com wrote: I believe there was a proposal internally that if the caller passes None for the callback, the RPC implementation should block until completion and then return the result. Does that seem like what you want? On Wed, Jan 7, 2009 at 6:23 PM, Alan Kligman alan.klig...@gmail.com wrote: I poked around a bit and the code above isn't correct (for a couple of reasons). But the question remains: should stubs be generated that return the response down the callstack instead of passing it through a callback function? On Jan 7, 5:59 pm, Alan Kligman alan.klig...@gmail.com wrote: Does it make sense to change stub.rpc_channel.CallMethod( method_descriptor, rpc_controller, request, method_descriptor.output_type._concrete_class, callback) to return stub.rpc_channel.CallMethod( method_descriptor, rpc_controller, request, method_descriptor.output_type._concrete_class, callback=lambda x:None) in python/google/protobuf/service_reflection.py? If, for example, I'm running in a single thread and my underlying channel is blocking, it's strange to pass a callback in when control would normally return to the calling method after CallMethod has completed synchronously. Thoughts? a --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: The way to set value to extension in Protocol buffers
To give the same answer as Alek with slightly different wording:
The accessors for extensions are analogous to the accessors for a normal
field. For a normal message-type field, there is no set accessor. There
is only mutable. For example:
// Proto
message Foo {
optional Bar bar = 1;
}
message Bar {
optional int32 i = 1;
}
// C++
Foo foo;
foo.mutable_bar()-set_i(1);
So with extensions, you should not use SetExtension() for embedded
messages. You should use MutableExtension():
f.MutableExtension(Bar::ck)-set_offset(100);
On Thu, Jan 8, 2009 at 11:32 PM, chongyc chongy...@gmail.com wrote:
I defined proto by following .
message Ch {
required int32offset = 1;
}
message Foo {
required string cmd = 1;
extensions 1000 to max;
}
message Bar {
extend Foo {
required Ch ck = 1000;
}
}
Then I generated c++ code from it.
It's OK
I am going to set value to extension ck()
I compiled c++ source following below.
term2relay::Foo f;
term2relay::Bar b;
term2relay::Ch c;
c.set_offset (100);
f.SetExtension(Bar::ck, c);
But compiling failed in source code line
f.SetExtension(Bar::ck, c);
How to do it ?
Please help me.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: POSIX long command line arguments
Well, hackiness usually leads to deeper problems. For example, if there's two ways to write the flag, then it's harder to grep for usages of it, particularly if you aren't actually aware that there are multiple ways to write it. I'm still leaning against this. Anyone else have an opinion? On Mon, Jan 12, 2009 at 1:52 PM, David Biesack david.bies...@sas.comwrote: As a user, I'm less concerned with the internal hackiness level, more so with the consistent interface :-) The code is clean enough to support both, it appears. I'd vote for it; I can contribute some code if you like. On Jan 12, 2:08 pm, Kenton Varda ken...@google.com wrote: Hmm, you're probably right that the arguments should have used hyphens rather than underscores. That said, we certainly can't just change it without breaking people, and accepting both seems hacky. On Mon, Jan 12, 2009 at 10:19 AM, David Biesack david.bies...@sas.com wrote: --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Problem to MergeFromCodedStream()
Read all the bytes into a buffer, then parse the buffer once you have the
whole thing.
On Mon, Jan 12, 2009 at 4:11 PM, chongyc chongy...@gmail.com wrote:
Thank you very much for your help.
I want parsing string 's' by reading each a byte from s.
What is alternative way that you can recommend to me.
On Jan 12, 11:28 am, chongyc chongy...@gmail.com wrote:
I write test code following below.
t2r::prot::Request msg0, msg2, msg3, msg4;
std::string s;
// set all fields of msg.
.
.
bool b;
b = msg0.SerializeToString(s); // OK
msg0.PrintDebugString();
b = msg2.ParseFromString(s);// OK
b = msg2.IsInitialized(); // OK
msg0.PrintDebugString();
for(int i= 0; i s.size(); ++i)
{
char c = s[i];
std::cout == i
=
std::endl;
google::protobuf::io::ArrayInputStream inp(c, 1);
google::protobuf::io::CodedInputStream decoder(inp);
b = msg3.MergeFromCodedStream(decoder); //
FAILED
msg3.PrintDebugString();
std::cout
std::endl;
b = msg4.MergePartialFromCodedStream
(decoder); // OK
msg4.PrintDebugString();
b = msg4.IsInitialized(); // FAILED
}
b = msg3.IsInitialized(); // FAILED
b = msg4.IsInitialized(); // FAILED
I want parsing string 's' by reading each a byte from s.
What is wrong.
Please help me.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: Problem to MergeFromCodedStream()
Send the size of the message first, then the data.
On Mon, Jan 12, 2009 at 4:34 PM, chongyc chongy...@gmail.com wrote:
When I send multiple messages, I should delimit them, and parse the
buffer.
What is the best way delimitting it that you can recommend me ?
On Jan 13, 8:18 am, Kenton Varda ken...@google.com wrote:
Read all the bytes into a buffer, then parse the buffer once you have the
whole thing.
On Mon, Jan 12, 2009 at 4:11 PM, chongyc chongy...@gmail.com wrote:
Thank you very much for your help.
I want parsing string 's' by reading each a byte from s.
What is alternative way that you can recommend to me.
On Jan 12, 11:28 am, chongyc chongy...@gmail.com wrote:
I write test code following below.
t2r::prot::Request msg0, msg2, msg3, msg4;
std::string s;
// set all fields of msg.
.
.
bool b;
b = msg0.SerializeToString(s); // OK
msg0.PrintDebugString();
b = msg2.ParseFromString(s);// OK
b = msg2.IsInitialized(); // OK
msg0.PrintDebugString();
for(int i= 0; i s.size(); ++i)
{
char c = s[i];
std::cout == i
=
std::endl;
google::protobuf::io::ArrayInputStream inp(c, 1);
google::protobuf::io::CodedInputStream decoder(inp);
b = msg3.MergeFromCodedStream(decoder);
//
FAILED
msg3.PrintDebugString();
std::cout
std::endl;
b = msg4.MergePartialFromCodedStream
(decoder); // OK
msg4.PrintDebugString();
b = msg4.IsInitialized(); // FAILED
}
b = msg3.IsInitialized(); // FAILED
b = msg4.IsInitialized(); // FAILED
I want parsing string 's' by reading each a byte from s.
What is wrong.
Please help me.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: POSIX long command line arguments
On Mon, Jan 12, 2009 at 5:27 PM, Alek Storm alek.st...@gmail.com wrote: I'm for changing it. Command line flags get deprecated in software all the time. Only for good reasons. This isn't a good enough reason to deprecate. So the options are between having just the underscore versions or accepting both underscores and hyphens. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Problem to MergeFromCodedStream()
On Mon, Jan 12, 2009 at 8:32 PM, Dave Bailey d...@daveb.net wrote: I agree, and it seems to be a question that comes up frequently in this forum, so maybe we should add a page to the Wiki that discusses how to send and receive a stream of protobuf (or any) messages. I did add some documentation on that here: http://code.google.com/apis/protocolbuffers/docs/techniques.html#streaming Things like run length encoding, magic bytes, checksums, record types - these are either highly desirable or absolutely necessary when streaming blocks of opaque binary data over a network connection, reading a sequence of them from a file, or whatever. I think there may be a misconception out there that libprotobuf somehow magically takes care of all those things. It seems to me that people need to conceive of a serialized protobuf object as the payload to a packet, and it is their job to design a packet header that describes the payload with sufficient information such that it can be extracted and dispatched to the appropriate handler. Well said. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Packed option for repeated fields?
Sorry, we're trying to get it implemented in Java and Python first. I'm reviewing the Java implementation today. (Protobuf development can be slow since no one works on it as their main project...) On Thu, Jan 15, 2009 at 12:23 PM, Caleb caleb.epst...@gmail.com wrote: On Dec 3 2008, 2:53 pm, Kenton Varda ken...@google.com wrote: On Wed, Dec 3, 2008 at 10:14 AM, gregg greggh...@gmail.com wrote: What's the status of the packed option for repeated fields? Is this still intended for a future protobuf release? I searched for packed and the last post in this group about it was in August. Coincidentally, I'm currently in the process of reviewing a change that implements this. It probably won't be in 2.0.3, but may reach SVN in a week or two. Bump. Any news on this? I don't see anything relevant in the public SVN repo. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
