[protobuf] Re: Java: how to use protobuf to send messages quickly over sockets?

2009-10-31 Thread Kenton Varda
On Sat, Oct 31, 2009 at 7:52 AM, Evan Jones wrote: > Hm. It is a little interesting that it would use so much more memory > than using Java serializable ... > Protocol buffers need to keep track of which fields are set. Currently this is done using a bool for each field. Perhaps it would be mo

[protobuf] Re: Java: how to use protobuf to send messages quickly over sockets?

2009-10-31 Thread Evan Jones
On Oct 30, 2009, at 22:24 , jta23 wrote: > The protobuf version of my code uses about 950MB of memory (the Java > Serializable version is only using around 650MB) and I had the java - > Xmx flag set too low; in reality protobuf is extremely fast compared > to Java Serializable: Hm. It is a little

[protobuf] Re: Java: how to use protobuf to send messages quickly over sockets?

2009-10-30 Thread Kenton Varda
Ah. So Java Serialization has no good reason to be slower... but apparently it is! Hah! :) On Fri, Oct 30, 2009 at 7:24 PM, jta23 wrote: > > I'm a bit embarrassed :) > > The protobuf version of my code uses about 950MB of memory (the Java > Serializable version is only using around 650MB) an

[protobuf] Re: Java: how to use protobuf to send messages quickly over sockets?

2009-10-30 Thread jta23
I'm a bit embarrassed :) The protobuf version of my code uses about 950MB of memory (the Java Serializable version is only using around 650MB) and I had the java - Xmx flag set too low; in reality protobuf is extremely fast compared to Java Serializable: Java Serializable: 12,000 msgs/sec Proto

[protobuf] Re: Java: how to use protobuf to send messages quickly over sockets?

2009-10-30 Thread Evan Jones
On Oct 29, 2009, at 16:45 , jta23 wrote: > and send them like this: > > byte size = (byte)msg.getSerializedSize(); > outputStream.writeByte(size); > outputStream.write(msg.toByteArray()); You should do the following instead: 1. Create a CodedOutputStream wrapping your OutputStream. 2. Use Coded

[protobuf] Re: Java: how to use protobuf to send messages quickly over sockets?

2009-10-29 Thread jta23
Thanks for the info. I thought the Java serialization metadata would have been large compared to the relevant data, but I guess not! On Oct 29, 5:42 pm, Kenton Varda wrote: > It sounds plausible.  There's no fundamental reason why protocol buffers > should be faster than Java serialization, at l

[protobuf] Re: Java: how to use protobuf to send messages quickly over sockets?

2009-10-29 Thread Kenton Varda
It sounds plausible. There's no fundamental reason why protocol buffers should be faster than Java serialization, at least for simple objects like yours composed of a set of primitive values. Since Java serialization is implemented by the VM, it can probably optimize better than protobufs can. H