Re: Data structures using protocol buffers

2008-10-23 Thread Dave Bailey
Sounds like fun. Just curious what is actually supposed to be encoded in the protobuf message, if the graph is too big to fit in memory? Are you sending just a single node, with its edges? It seems like you wouldn't be able to avoid scenarios where some of the edges in your message point to

Re: RESTful queries (GET) with Protocol Buffers

2008-11-25 Thread Dave Bailey
I was actually doing something similar recently. I am not an expert on web services or any of that, but I am using GET, with the serialized message encoded as a URL-safe base64 variant (using '-' instead of '+', and '_' instead of '/'). This makes it so that every request can be represented as

Re: Slicing support in Python

2008-12-03 Thread Dave Bailey
On Dec 2, 10:49 pm, Kenton Varda [EMAIL PROTECTED] wrote: C++ compatibility matters because eventually we want to be able to generate Python code which just wraps C++ code for efficiency.  C++ isn't garbage collected, so append() can't easily be implemented in this case without having

Re: POSIX long command line arguments

2009-01-12 Thread Dave Bailey
It would lead to an inconsistency with all of the other compiler options that adhere to the underscore convention: --decode_raw --descriptor_set_out --disallow_services --include_imports --proto_path I think either all or none of them should be changed. -dave On Jan 12, 2:48 pm, Kenton Varda

Re: POSIX long command line arguments

2009-01-12 Thread Dave Bailey
(Personally, I prefer the GNU option format, so my vote is that if a patch that passes muster is provided, it should be used). -dave On Jan 12, 3:11 pm, Dave Bailey d...@daveb.net wrote: It would lead to an inconsistency with all of the other compiler options that adhere to the underscore

Re: Isn't the choice of std::string (C++ API) as a container for binary serialized bytes dangerous b/c of null termination?

2009-02-25 Thread Dave Bailey
Kenton, The generated code for a string field has setters for std::string and const char *, but do you think you could add a (const char *, int length) setter as well (like you have for bytes fields)? Right now in protobuf-perlxs, we have to construct a temporary std::string in order to set a

Re: Performance comparison of Thrift, JSON and Protocol Buffers

2009-03-03 Thread Dave Bailey
Justin, Thanks for writing this up; I think it's a nice real world example. I ran an equivalent test (using your same .proto files) in Perl to compare JSON::XS, protobuf-perlxs, and Storable. I did this on an x86_64 quad-core Xeon (2.5 GHz) and found: 1) Your original dns.proto (with

Re: build fix for protobuf-c

2009-03-03 Thread Dave Bailey
We use a --with-protobuf option in protobuf-perlxs. http://code.google.com/p/protobuf-perlxs/source/browse/trunk/configure.ac I actually copy-pasted the configure.ac part that implements --with- protobuf into the protobuf-c configure.ac so I could build protobuf-c in my environment recently.

Re: optimize_for option default

2009-03-05 Thread Dave Bailey
+1 for SPEED. -dave On Mar 5, 3:39 pm, Kenton Varda ken...@google.com wrote: Hi all, As you know if you've read the docs carefully, when using C++ or Java protocol buffers, for best performance you need to add a line to your .proto files:   option optimize_for = SPEED; Otherwise, by

Re: python performance

2009-03-20 Thread Dave Bailey
If you are able to use the other P (Perl) instead of Python or PHP, http://code.google.com/p/protobuf-perlxs will give performance comparable to PHP JSON, I would bet. -dave On Mar 20, 11:59 am, dvirsky dvir...@gmail.com wrote: Thanks, It seems as though the serialization is the slowest part,

abort() in libprotobuf due to missing required field

2009-03-23 Thread Dave Bailey
Hi, libprotobuf calls abort() if one attempts to serialize a message that is missing a required field. To me, this seems excessive, because it gives the calling code no opportunity to handle the error in a different way. Here is the bug report in protobuf-perlxs:

Re: abort() in libprotobuf due to missing required field

2009-03-23 Thread Dave Bailey
this with an exception.  Is -DNDEBUG good enough for you or do you want to look for another solution? On Mon, Mar 23, 2009 at 2:32 PM, Dave Bailey d...@daveb.net wrote: Hi, libprotobuf calls abort() if one attempts to serialize a message that is missing a required field. To me

Re: Streaming different types of messages

2009-03-27 Thread Dave Bailey
Kenton, I don't suppose there'd ever be a way to mark a set of fields as mutually exclusive? -dave On Mar 27, 8:42 am, Jon Skeet sk...@pobox.com sk...@pobox.com wrote: On Mar 27, 1:32 pm, achin...@gmail.com wrote: If I understand correctly there is no good way to use proto buffers to

[protobuf] Re: Options

2010-03-09 Thread Dave Bailey
I think it would be really helpful to be able to override anything in FileOptions from the command line, at least. I'm not sure if anyone would really want to override options at other levels (Message/Field/ Enum/Service/Method)... but maybe something like: # apply a file-level option to

[protobuf] Re: Variant-length encoding : is it OK to be wasteful?

2010-03-10 Thread Dave Bailey
I imagine there would be some pathological cases where the same memory would get repeatedly shifted to the right by 1 byte at a time as the parent message length prefixes get resized. It seems like the explicit precomputation of lengths is likely to be more efficient overall... that said, maybe

[protobuf] Re: Why to reinvent the wheel ?

2010-11-09 Thread Dave Bailey
On Nov 9, 6:15 am, Kalki70 kalki...@gmail.com wrote: Hello again, On Nov 9, 2:59 am, Kenton Varda ken...@google.com wrote: [...] The bigger problem with ASN.1, though, is that it is way over-complicated.  It has way too many primitive types.  It has options that are not needed.  The

Re: [protobuf] new code generator: protobuf-nginx

2013-04-22 Thread Dave Bailey
The generated code relies on nginx-specific types such as its typedefs for strings, arrays, and rbtrees, and also its memory pools and so on. The generated code is meant to be compiled into the nginx binary, which is currently the way all module development is done with nginx. -dave On Apr

Re: [protobuf] new code generator: protobuf-nginx

2013-04-22 Thread Dave Bailey
are ok with it. On Mon, Apr 22, 2013 at 11:28 AM, Dave Bailey d...@daveb.net wrote: The generated code relies on nginx-specific types such as its typedefs for strings, arrays, and rbtrees, and also its memory pools and so on. The generated code is meant to be compiled into the nginx binary