Re: [protobuf] Static allocation

2012-07-19 Thread Evan Jones
On Jul 18, 2012, at 16:14 , Jeremy wrote: I understand, but if one wants to keep a large persistent message allocated and walk over it frequently, there is a price to pay on cache misses that can be significant. I guess you are wishing that the memory layout was completely contiguous? Eg.

Re: [protobuf] Static allocation

2012-07-17 Thread Evan Jones
On Jul 17, 2012, at 2:33 , Jeremy Swigart wrote: Is there a way to tell the proto compiler to generate message definitions for which the message fields are statically defined rather than each individual field allocated with dynamic memory? Obviously the repeater fields couldn't be fully

Re: [protobuf] where is input_stream.py?

2012-07-16 Thread Evan Jones
On Jul 14, 2012, at 10:55 , jrf wrote: Is there a reason that a python equivalent of CodedInputStream is not part of protobuf? I seem to recall that the answer is basically yeah, it probably should be but no one really works on this stuff any more. You can dig around in the

Re: [protobuf] where is input_stream.py?

2012-07-16 Thread Evan Jones
On Jul 16, 2012, at 14:43 , jrf wrote: Is that because protobufs is done or not being further developed? You would need to get someone from Google to answer. The impression I get is that the open source release is, at the very least, in maintenance mode where they occasionally fix bugs etc.

Re: [protobuf] Error while using parseFrom

2012-06-27 Thread Evan Jones
On Jun 26, 2012, at 11:08 , d34th4ck3r wrote: What is it that I am doing wrong? Protocol buffers are a *binary* format. Those funny characters at the end of the string are probably part of the message, and you should leave them there. You also should not be passing them around as strings. They

Re: [protobuf] Best practices for proto file organization in large projects

2012-06-19 Thread Evan Jones
On Jun 18, 2012, at 22:49 , Justin Muncaster wrote: I'm currently changing our build system to be cmake based and I'm again finding myself fighting with the build system to get the .proto to be automatically generated in a way where they build correctly. What specific problems are you

Re: [protobuf] 1MB message limit (recommendation)

2012-05-30 Thread Evan Jones
On May 29, 2012, at 23:26 , msrobo wrote: According to the documentation, it's recommended that the message size be = 1 Megabyte. I've searched around for the reason for this recommendation, but I can't seem to find anything. Based on some basic benchmarking serializing/unserializing messages

Re: [protobuf] message member case problem

2012-05-16 Thread Evan Jones
On May 16, 2012, at 5:02 , secondsquare wrote: After generating cpp files, the member becomes msgsize. Big ‘S’ is changed to little 's'. This is by design. Protocol buffers follows Google's style guide where C++ names_use_underscores while Java names useCamelCase. Protobuf will generate the

Re: [protobuf] incompatible type changes philosophy

2012-05-10 Thread Evan Jones
On May 9, 2012, at 15:26 , Jeremy Stribling wrote: * There are two nodes, 1 and 2, running version A of the software. * They exchange messages containing protobuf P, which contains a string field F. * We write a new version B of the software, which changes field F to an integer as an

Re: [protobuf] incompatible type changes philosophy

2012-05-09 Thread Evan Jones
On May 8, 2012, at 21:26 , Jeremy Stribling wrote: Thanks for the response. As you say, this solution is painful because you can't enable the optimization until the old version of the program is completely deprecated. This is somewhat simple in the case that you yourself are deploying the

Re: [protobuf] Protocol Buffers for IOS

2012-03-31 Thread Evan Jones
On Mar 31, 2012, at 4:31 , Dhanaraj G wrote: I have gone through he following link.. http://code.google.com/p/metasyntactic/wiki/ProtocolBuffers There is no official support but I've used the following distribution with success, with the latest protoc (I'm pretty sure):

Re: [protobuf] [Java][Python][TCP] Reading messages that where written with writeDelimited and viceversa

2012-03-27 Thread Evan Jones
On Mar 26, 2012, at 21:49 , Galileo Sanchez wrote: Thanks man... It worked great! I guess I should read the documentation a little more xP. Sadly these functions aren't actually documented. The Python API doesn't expose these routines for some reason I don't understand / remember. Glad it

Re: [protobuf] Problem with accent

2012-03-26 Thread Evan Jones
On Mar 23, 2012, at 9:07 , Simon wrote: I have an annoying problem with some accent. I build my proto-object, no problem, and when i want to read it the browser, using .toString function, i have \303\240 instead of à, \303\250 instead of è, etc… What do you mean i want to read it the browser

Re: [protobuf] [Java][Python][TCP] Reading messages that where written with writeDelimited and viceversa

2012-03-26 Thread Evan Jones
On Mar 25, 2012, at 18:09 , Galileo Sanchez wrote: else if (Should I write the size as a raw bit string?) thenHow do I do that? You need to use something like the following. Not 100% sure it works but it should be close? Hope this helps, Evan # Output a message to

Re: [protobuf] Protocol Buffers for version control of objects on a cache.

2012-03-21 Thread Evan Jones
On Mar 20, 2012, at 16:12 , Mick wrote: These objects are going to be accessible to multiple users, who's accessor programs may be on different release cycles. I have been looking into protocol buffers as a way of managing dataloss/corruption between versions. Has anyone used protocol

Re: [protobuf] Re: How to read continuous stream of messages from TCP

2012-03-08 Thread Evan Jones
On Mar 8, 2012, at 2:30 , waynix wrote: Since this is so common an issue and the suggested solution is almost de facto standard, (saw this after my initial post: http://code.google.com/apis/protocolbuffers/docs/techniques.html), it begs the question of why not build it into protobuf proper.

Re: [protobuf] How to read continuous stream of messages from TCP

2012-03-06 Thread Evan Jones
On Feb 27, 2012, at 17:27 , waynix wrote: 1. Is this still the way to do it? Seems quite cumbersome (to lazy me ;-). Is there a wrapper built in to do this? Yes. Sadly there is no wrapper included in the library. 2. If I understand Jason's suggestion riht, the length is really not part

Re: [protobuf] Message thread safety in Java

2012-02-20 Thread Evan Jones
On Feb 20, 2012, at 8:25 , Frank Durden wrote: I'm sorry if this is explained somewhere, I couldn't find an answer. Are protobuf messages (in Java) thread safe for concurrent reads. I guess they're immutable in the sense that you can't modify them after they're built, but can a message object

Re: [protobuf] Message thread safety in Java

2012-02-20 Thread Evan Jones
On Feb 20, 2012, at 16:20 , Christopher Smith wrote: Message objects *don't* have mutators and are conceptually a copy of the relevant builder object. Having attempted to refresh my knowledge of the Java Memory Model, I think there is a subtle difference between an object that has all final

Re: [protobuf] Error: Byte size calculation and serialization were inconsistent

2012-02-07 Thread Evan Jones
On Feb 6, 2012, at 21:54 , Robby Zinchak wrote: It turned out to be an uninitialized boolean. Properly setting the value in question seems to allow things to proceed normally. Ah! Interesting. So one of your .set_* properties is a boolean, and one of them was uninitialized? That would do it.

Re: [protobuf] Error: Byte size calculation and serialization were inconsistent

2012-02-06 Thread Evan Jones
This is weird. I don't see any clear potential cause, so I have a few questions: HTMud::EnvAdd item; item.set_id(ID); item.set_idtype(typeID); item.set_x(X); item.set_y(Y); item.set_z(Z); item.set_lockdown(lockdown); item.set_mapid(map); item.set_tilesetno(tilesetNo);

Re: [protobuf] Re: Problem with C++ -writing multiple messages with a repeated field to a file

2011-05-16 Thread Evan Jones
On May 16, 2011, at 9:45 , Nigel Pickard wrote: I have actually got the code working, but it involves creating a new output stream everytime I write to it (surely got to be wasteful and not the right way?). Definitely not needed, and it will be more efficient if you can re-use a single

Re: [protobuf] Problem with C++ -writing multiple messages with a repeated field to a file

2011-05-13 Thread Evan Jones
On May 13, 2011, at 10:12 , Nigel Pickard wrote: libprotobuf FATAL google/protobuf/io/zero_copy_stream_impl_lite.cc: 346] CHECK failed: (buffer_used_) == (buffer_size_): BackUp() can only be called after Next(). Off the top of my head, I *believe* this is happening because the

Re: [protobuf] on the wire sizes

2011-04-01 Thread Evan Jones
On Apr 1, 2011, at 6:54 , AdrianPilko wrote: What is the [best] way to determine the on the wire size? You probably want msg.ByteSize() in C++, msg.getSerializedSize() in Java. Evan -- http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups

Re: [protobuf] A protocol message was rejected because it was too big ???

2011-03-07 Thread Evan Jones
On Mar 6, 2011, at 18:45 , ksamdev wrote: I think I found the source of the problem. The problem is that CodedInputStream has internal counter of how many bytes are read so far with the same object. Ah, right. With the C++ API, the intention is that you will not reuse the

Re: [protobuf] A protocol message was rejected because it was too big ???

2011-03-07 Thread Evan Jones
On Mar 7, 2011, at 13:03 , ksamdev wrote: Hmm, thanks for the advice. It may work fine. Nevertheless, I have to skip previously read messages in this case every time CodedInputStream is read. Not true: Creating a CodedInputStream does not change the position in the underlying stream. Your

Re: [protobuf] How to get the byte[] from a serialized data ?

2011-03-04 Thread Evan Jones
On Mar 4, 2011, at 7:15 , Aditya Narayan wrote: I have created .proto files and compiled them to get the generated classes. Also I can build the message objects using the setters finally build() method. But to store it to database, I need serialized data as byte[] or byte buffers. How do I

Re: [protobuf] Chunking a large message

2011-03-04 Thread Evan Jones
On Mar 3, 2011, at 15:53 , Linus wrote: I am wondering if there are any examples of chunking large PB messages (about 1MB) into smaller chunks, to transmit over the wire. This is going to be pretty application specific. Typically it involves taking one message with a huge repeated field and

Re: [protobuf] RuntimeException while parsing back the byte[] to protocol buffer message instance! (deserialization)

2011-03-04 Thread Evan Jones
On Mar 4, 2011, at 11:11 , Aditya Narayan wrote: Exception in thread main java.lang.RuntimeException: Uncompilable source code This error means there is a build problem in your Eclipse project. You are trying to call some code that is not building compiled correctly. Fix your

Re: [protobuf] Can a message derive from another message?

2011-03-03 Thread Evan Jones
On 03/02/2011 10:04 AM, ZHOU Xiaobo wrote: required string Content = 3; WARNING: You should be using type bytes here, not type string. This doesn't matter for C++, but matters for other languages which will assume strings contain UTF-8 data. Evan -- http://evanjones.ca/ -- You

Re: [protobuf] Beginner's Q: Does protobuf generate underlying transport sockets as well

2011-02-28 Thread Evan Jones
On Feb 28, 2011, at 11:46 , footloose wrote: The tutorials talk only about marshalling and un marshalling the data structures. Do the sockets have to be written manually? Yes. The protocol buffer library from Google does not include an RPC implementation. There are a bunch of third-party

Re: [protobuf] Fwd: RpcChannel and RpcController Implementation

2011-02-21 Thread Evan Jones
On Feb 21, 2011, at 3:06 , Amit Pandey wrote: Did anyone get the chance to look into it. If you want to use the RPC system, you need to provide your own implementation, or maybe use an existing one, such as: http://code.google.com/p/protobuf/wiki/ThirdPartyAddOns#RPC_Implementations If

Re: [protobuf] New protobuf feature proposal: Generated classes for streaming / visitors

2011-02-08 Thread Evan Jones
I read this proposal somewhat carefully, and thought about it for a couple days. I think something like this might solve the problem that many people have with streams of messages. However, I was wondering a couple things about the design: * It seems to me that this will solve the problem

Re: [protobuf] New protobuf feature proposal: Generated classes for streaming / visitors

2011-02-08 Thread Evan Jones
On Feb 8, 2011, at 13:34 , Kenton Varda wrote: I handle user messages by passing them as bytes, embedded in my own outer message. This is what I do as well, as does protobuf-socket-rpc: http://code.google.com/p/protobuf-socket-rpc/source/browse/trunk/proto/rpc.proto I guess I was thinking

Re: [protobuf] Re: protobuf not handling special characters between Java server and C++ client

2011-01-26 Thread Evan Jones
On Jan 26, 2011, at 3:43 , Hitesh Jethwani wrote: Can we encode the protobuf data in ISO-8859-1 from the server end itself? Yes. In this case, you need to use the protocol buffer bytes type instead of the protocol buffer string type, since you want to exchange ISO-8859-1 bytes from

Re: [protobuf] protobuf not handling special characters between Java server and C++ client

2011-01-25 Thread Evan Jones
On Jan 25, 2011, at 15:27 , Hitesh Jethwani wrote: As may be evident from above I am naive at Java and Protobuf. Any help on this is appreciated. The Java protocol buffer API encodes strings as UTF-8. Since C++ has no unicode support, what you get on the other end is the raw UTF-8

Re: [protobuf] protocol buffers and client-server communication

2011-01-23 Thread Evan Jones
message in the stream?. See: http://code.google.com/apis/protocolbuffers/docs/techniques.html#union the archives of this group also contain many discussions on this subject. Evan Jones -- http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol

Re: [protobuf] Dealing with Corrupted Protocol Buffers

2011-01-20 Thread Evan Jones
of a pain. Good luck, Evan Jones -- http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr

Re: [protobuf] custom constructor

2011-01-14 Thread Evan Jones
work, but it works. Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr

Re: [protobuf] Re: Large message vs. a list message for smaller messages

2011-01-13 Thread Evan Jones
with the framework you are using, but this should be feasible. Hope this helps, Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com

Re: [protobuf] Java Newbie Question: Attaching a CodedInputStream to an NIO Socket

2011-01-13 Thread Evan Jones
/protorpc/ProtoConnection.java#l40 Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com. To unsubscribe from this group, send email to protobuf

Re: [protobuf] Java Newbie Question: Attaching a CodedInputStream to an NIO Socket

2011-01-12 Thread Evan Jones
) method of this class, so it is actually pretty straightforward. There might be a better way but it works for me. Hope this helps, Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group

Re: [protobuf] Large message vs. a list message for smaller messages

2011-01-12 Thread Evan Jones
are really sending / writing a large number of messages, you want to read something like: http://code.google.com/apis/protocolbuffers/docs/techniques.html#streaming Good luck, Evan Jones -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google

Re: [protobuf] Using a ByteBuffer instead of a ByteString?

2011-01-11 Thread Evan Jones
to worry about the ByteBuffer being accidentally changed, etc). The latest version of Protocol Buffers in Subversion has ByteString.copyFrom(ByteBuffer) which will do what you want efficiently. Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed

Re: [protobuf] Protocol Buffers Python extension in C

2011-01-08 Thread Evan Jones
he was interested in the same problem. Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email

Re: [protobuf] Re: java parse with class known at runtime (and compiled proto)

2010-12-06 Thread Evan Jones
Hope this helps, Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr

Re: [protobuf] java parse with class known at runtime (and compiled proto)

2010-12-03 Thread Evan Jones
(); } You can get defaultInstance from ConcreteMessageType.getDefaultInstance(); You may want to create a tiny InputStream wrapper around ByteBuffer to avoid an extra copy, or if you know it is a heap byte buffer, use the array mergeFrom(). Hope that helps, Evan -- Evan Jones http

Re: [protobuf] 2.4.0 and lazy UTF-8 conversions in Java

2010-12-01 Thread Evan Jones
, although I don't think I ever considered this particular version. However, I think this can be made thread-safe, even without volatile (although I only understand the JMM enough to be dangerous). Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed

Re: [protobuf] 2.4.0 and lazy UTF-8 conversions in Java

2010-11-30 Thread Evan Jones
already have both byte[] and String fields for each string due to an encoding improvement I contributed, so this should be nearly a pure win. Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post

Re: [protobuf] fails to parse from string

2010-11-10 Thread Evan Jones
a length. See: http://code.google.com/apis/protocolbuffers/docs/techniques.html#streaming Or search the group archives for threads such as: http://groups.google.com/group/protobuf/browse_thread/thread/3af587ab16132a3f Good luck, Evan -- Evan Jones http://evanjones.ca/ -- You received

Re: [protobuf] fails to parse from string

2010-11-10 Thread Evan Jones
are using UDP, it will end up not working as soon as you have a message which is bigger than either your buffer, or the maximum UDP packet size, whichever comes first. Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol

Re: [protobuf] help required

2010-11-03 Thread Evan Jones
these objects. See: http://code.google.com/apis/protocolbuffers/docs/javatutorial.html Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com

Re: [protobuf] CodedInputStream on top of sockets

2010-11-02 Thread Evan Jones
. This is typically easier since CopyingInputStreamAdaptor then implements the appropriate buffering logic. See: http://code.google.com/apis/protocolbuffers/docs/reference/cpp/google.protobuf.io.zero_copy_stream_impl_lite.html#CopyingInputStreamAdaptor Good luck, Evan -- Evan Jones http

Re: [protobuf] Generic Message Dispatch and Message Handler

2010-10-27 Thread Evan Jones
only need to choose between a limited set of types, you may want a union type or extensions instead: http://code.google.com/apis/protocolbuffers/docs/techniques.html#union http://code.google.com/apis/protocolbuffers/docs/proto.html#extensions Evan -- Evan Jones http://evanjones.ca/ -- You

Re: [protobuf] Re: Generic Message Dispatch and Message Handler

2010-10-27 Thread Evan Jones
://people.csail.mit.edu/evanj/hg/index.cgi/javatxn/file/tip/src/ca/evanjones/protorpc/ServiceRegistry.java http://people.csail.mit.edu/evanj/hg/index.cgi/javatxn/file/tip/src/ca/evanjones/protorpc/ProtoMethodInvoker.java Good luck, Evan -- Evan Jones http://evanjones.ca/ -- You received this message because

Re: [protobuf] protocol buffer within a protocol buffer from C++ to Java

2010-10-26 Thread Evan Jones
On Oct 25, 2010, at 21:45 , Paul wrote: optional string meas_rec_str = 2; Change this to: optional bytes meas_rec_str = 2; Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group

Re: [protobuf] Re: Message missing required fields exception when parsing a message that has required fields defaulted

2010-10-26 Thread Evan Jones
are serialized differently. Hope this helps, Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email

Re: [protobuf] ParseFromArray -- in Java

2010-10-25 Thread Evan Jones
http://www.javapractices.com/topic/TopicAction.do?Id=29) That said, I think you *might* be able to hack something like this using the Builder object. I would be interested to know if you try this, and if it has any performance benefits. Evan -- Evan Jones http://evanjones.ca/ -- You

Re: [protobuf] Re: Delay in Sending Data

2010-10-22 Thread Evan Jones
as well. Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr

Re: [protobuf] Delay in Sending Data

2010-10-21 Thread Evan Jones
for many conversations about this. Hope this helps, Evan http://code.google.com/apis/protocolbuffers/docs/techniques.html#streaming -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group

Re: [protobuf] buffer sizes when sending messages from c++ to java

2010-10-20 Thread Evan Jones
using the protobuf API: it handles any required byte swapping for you. Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from

Re: [protobuf] Re: valgrind invalid write and double free errors

2010-10-14 Thread Evan Jones
if that is getting called multiple times? This FileDescriptorTable object is used internally by the protobuf library and I don't really understand it. I'm hoping someone who might understand this code might be able to suggest where this double free could be coming from. Evan -- Evan

Re: [protobuf] sending a message over TCP from a C++ client to a Java server

2010-10-13 Thread Evan Jones
: codedOutput.WriteVarint32(msg.ByteSize()); msg.SerializeToCodedStream(codedOutput); codedOutput.flush(); ... Hope this helps, Evan (as an aside: the C++ API really should have an equivalent to writeDelimitedTo and parseDelimited on the Java side). -- Evan Jones http://evanjones.ca/ -- You received

Re: [protobuf] Re: sending a message over TCP from a C++ client to a Java server

2010-10-13 Thread Evan Jones
) are coming out the other end and getting passed into parseDelimited. It might be helpful if you sent a snippet of code where you are sending and receiving the messages, but I can't think of anything off the top of my head. Evan -- Evan Jones http://evanjones.ca/ -- You received this message

Re: [protobuf] valgrind invalid write and double free errors

2010-10-13 Thread Evan Jones
. The FileDescriptorTables are static objects of sorts, I think. Are you calling ShutdownProtobufLibrary() somewhere? Maybe more than once? Memory leaks *will* be reported by valgrind if you don't call ShutdownProtobufLibrary(), but I don't know what could cause a double free. Evan -- Evan Jones

Re: [protobuf] Feature proposal: mapped fields

2010-10-06 Thread Evan Jones
rather than a struct or container type. There are some languages which do not support more complex key types in their native map types. In addition the JSON protocol only supports key types that are base types. Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you

Re: [protobuf] Timeouts for reading from a CodedInputStream

2010-09-28 Thread Evan Jones
I never call the CodedInputStream routines unless I know there is sufficient data. This may not be ideal for your application, so your milage may vary. Good luck, Evan Jones -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups

Re: [protobuf] Re: Timeouts for reading from a CodedInputStream

2010-09-28 Thread Evan Jones
it is convenient. Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr

Re: [protobuf] Re: MyType_Parse() calls ParseNamedEnum() with 'const std::string' parameter instead of 'const string'

2010-09-22 Thread Evan Jones
is in the google::protobuf::internal namespace, rather than in google::protobuf? Good luck, Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com

Re: [protobuf] Re: Silly beginner question: Do the different RPC implementations inter-work?

2010-08-27 Thread Evan Jones
service. Were I to start it today, I would use the code generator, since there are a few small things in the automatically generated RPC interface that I would like to change. Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups

Re: [protobuf] Status of protobufs

2010-08-26 Thread Evan Jones
) http://code.google.com/apis/protocolbuffers/docs/reference/java/com/google/protobuf/MessageLite.Builder.html#mergeDelimitedFrom(java.io.InputStream) Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group

Re: [protobuf] Performance of java proto buffers

2010-08-22 Thread Evan Jones
that improves Java message *encoding* performance, by optimizing string encoding. It is available at the following URL. Unfortunately, there is no similar approach to improving the decoding performance. http://codereview.appspot.com/949044/ Evan -- Evan Jones http://evanjones.ca/ -- You

Re: [protobuf] Service can only receive one argument

2010-08-22 Thread Evan Jones
#union Two RPCs: service Service { rpc sendObject1(Object1) returns (Result1); rpc sendObject2(Object2) returns (Result2); } Hope this helps, Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post

Re: [protobuf] Re: How to retrieve parameters using tag numbers using CodedInputStream

2010-08-16 Thread Evan Jones
message. However, I suspect that isn't really a big overhead. You can also use YourProtocolMessage.getDefaultInstance() to avoid creating a message. Hope this helps, Evan Jones -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups

Re: [protobuf] Re: Java implementation questions

2010-08-05 Thread Evan Jones
and fill in the length field. This would be an incompatible change to the serialization format, however. Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto

Re: [protobuf] Total bytes limit on coded input stream in C++?

2010-08-04 Thread Evan Jones
, or MessageLite:: ParseFromBoundedZeroCopyStream). Good luck, Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send

Re: [protobuf] Setting a nested message field without making a pointer?

2010-08-04 Thread Evan Jones
On Aug 4, 2010, at 9:50 , mark.t.macdon...@googlemail.com wrote: Is there a way I can do this without creating a Pixel pointer? Something like this (which doesn't compile): Try fractal.mutable_pixel(0)-set_* Hope this helps, Evan -- Evan Jones http://evanjones.ca/ -- You received

Re: [protobuf] Total bytes limit on coded input stream in C++?

2010-08-03 Thread Evan Jones
/browse_thread/thread/a4bc2a3788d356f6 Read that thread for details, but the summary is: patches welcomed. CodedInputStream is pretty lightweight though, so creating and destroying one per message should be pretty efficient. Evan -- Evan Jones http://evanjones.ca/ -- You received this message

Re: [protobuf] Cannot parse message with CodedInputStream over a pipe

2010-07-30 Thread Evan Jones
is what that method does under the covers (I think). Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send

Re: [protobuf] Error in repeated_field.h

2010-07-30 Thread Evan Jones
you have memory corruption somewhere? Try using valgrind if you might have memory corruption. Good luck, Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto

Re: [protobuf] Protocol Buffers RPC design questions

2010-07-16 Thread Evan Jones
but the 'EnhancedRpcController' then becomes a pervasive cast as well as a RPC implementation lockin This is probably the worst part of the built-in service API, in my opinion. I end up with casts related to controllers in lots of places. Its ugly, but I don't see any good way to fix it. Evan -- Evan

Re: [protobuf] Help with basic concepts of descriptors and reflection

2010-07-16 Thread Evan Jones
(effectively). The second has to do some sort of table lookup. You want to use the first form if you care about performance. Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email

Re: [protobuf] Re: C++ syntax: how to set a singular enum field

2010-07-16 Thread Evan Jones
be generated. Note that the .proto you sent doesn't have RETRACTED defined, so maybe that is your problem? Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto

Re: [protobuf] Protobuf for client/server messaging?

2010-07-14 Thread Evan Jones
/Descriptors.Descriptor.html#getFullName() Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf

Re: [protobuf] Can't read in Java message sent from C++

2010-07-10 Thread Evan Jones
luck, Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com

Re: [protobuf] Basic message encoding/decoding

2010-07-07 Thread Evan Jones
/ CodedOutputStream is probably helpful. Also: did you look at the third party libraries? Many programming languages have implementations you could try using: http://code.google.com/p/protobuf/wiki/ThirdPartyAddOns Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you

Re: [protobuf] Serialized Message Field Order

2010-07-06 Thread Evan Jones
); -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more

Re: [protobuf] Detecting end of CodedInputStream

2010-06-25 Thread Evan Jones
(success); This works for me. Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf

Re: [protobuf] using compression on protobuf messages

2010-06-22 Thread Evan Jones
as they are written out, rather than needing to serialize the entire thing to a byte[] array, then compress it. This could be better, but as always you'll have to measure it. Hope this helps, Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed

Re: [protobuf] using compression on protobuf messages

2010-06-22 Thread Evan Jones
message.writeTo(servletOutputStream) If you are writing multiple messages, you'll probably want to explicitly create a single CodedOutputStream to write all of them. If you experiment with this and find something different, I would be interested to know. Evan -- Evan Jones http://evanjones.ca

Re: [protobuf] Re: using compression on protobuf messages

2010-06-22 Thread Evan Jones
. This is trickier than re-using a single CodedOutputStream. If you are interested, I can send the details about what I have used. Although to be honest: I haven't tested it carefully to see if this is *actually* faster than doing the simple thing such as .parseFrom() and friends. Evan -- Evan Jones http

Re: [protobuf] Re: Efficiently reading/writing multiple messages in C++

2010-06-19 Thread Evan Jones
be opposed to putting some effort into including it in protobuf, or in a protobuf-utils type project. Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto

[protobuf] Re: Efficiently reading/writing multiple messages in C++

2010-06-17 Thread Evan Jones
On Jun 17, 2010, at 17:05 , Evan Jones wrote: I'm working around this by moving the CodedInputStream inside my loop, which is fine, but seems sub-optimal. At the very least, since I have lots of small messages, this means my ZeroCopyInputStream's methods get called many times. Based

[protobuf] String Encoding Performance Improvement

2010-06-06 Thread Evan Jones
in 32.337s; 15.353779MB/s -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr

Re: [protobuf] Issues with Large Coded Stream Files?

2010-06-03 Thread Evan Jones
call .resetSizeCounter() occasionally, I think it should work just fine. I'm certainly using a single Java CodedInputStream per long lived connection without any trouble. Unclear if I've sent 2GB of data over a single connection though. Evan -- Evan Jones http://evanjones.ca/ -- You

Re: [protobuf] Issues with Large Coded Stream Files?

2010-06-03 Thread Evan Jones
? No, this should work just fine. On the input size, you'll need to call CodedInputStream.resetSizeCounter() after each message, otherwise you'll run into the size limit. Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers

Re: [protobuf] Re: Implementing protobuf in symbian

2010-06-02 Thread Evan Jones
this helps, Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com

Re: [protobuf] Java UTF-8 encoding/decoding: possible performance improvements

2010-06-01 Thread Evan Jones
a ThreadLocal? From memory, the ThreadLocal appears to be very cheap, and not make much performance difference, but I should double check this as well. Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group

Re: [protobuf] Java UTF-8 encoding/decoding: possible performance improvements

2010-05-31 Thread Evan Jones
others are interested. Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr

Re: [protobuf] Java UTF-8 encoding/decoding: possible performance improvements

2010-05-30 Thread Evan Jones
there we should be able to decode directly from the input buffer, saving a copy and an allocation of a temporary byte[] array. Unclear if this will actually be significantly faster, but it might be slightly faster. Evan -- Evan Jones http://evanjones.ca/ -- You received this message

  1   2   >